Evaluating Fidelity in Synthetic Tabular Data Generation: A Comparative Study of CTGAN and TVAE for Human Activity Recognition Datasets

The challenges associated with collecting real-world data are largely addressed through the generation of Synthetic Data (SD) across various domains. Utility, fidelity and privacy represent the key challenges in the synthetic tabular data generation, and each of these offers a unique perspective. In this research, we focused on the fidelity of the generated tabular data in comparison to real data, using four main metrics recommended in previous literature: Hellinger Distance (HD), Pairwise Correlation Differences (PCD), R-squared Depth vs. Depth (R2DD) Plot, and Area Under Receiver Operating Characteristic Curve (AUC-ROC). We used two Human Activity Recognition (HAR) datasets, 1) Mobile Health (mHealth) and 2) HAR Using Smartphones (HARUS); these datasets differ in the number of activities and sample sizes. We generated data using two generative methods: Conditional Tabular Generative Adversarial Networks (CTGAN) and Tabular Variational Autoencoders (TVAE).

The early results indicate that CTGAN achieved a 45% lower HD on mHealth (0.0608 vs. 0.1100), while TVAE achieved a 32% lower HD on HARUS (0.0825 vs. 0.1212). CTGAN excelled on mHealth at 1500/500 epochs (PCD 0.0295, R2DD Plot 0.9855, AUC-ROC 0.5591), whereas TVAE excelled on HARUS at 1800/700 epochs (PCD 0.0639, R2DD Plot 0.5213, AUC-ROC 0.6424). These findings suggest that adjusting the generative technique according to dataset characteristics, such as sample size and feature complexity, is useful. Future work will expand this analysis by integrating additional generative methods and datasets to explore the utility and privacy of synthetic data with fidelity.