Privacy through Synthesis: Exploration of Tabular Synthetic Data Generation Methods and Quality Assessment

Privacy-related ML attacks

a.

Adversarial knowledge can be used to model the various attack surfaces against ML models [10]. The scope of knowledge ranges from having access to the API of ML to being fully aware of model parameters and the training settings. A variety of scenarios fall somewhere in the middle, such as having little knowledge of training configuration, hyperparameters, and the model’s architecture[10]. In attacks pertaining to privacy, the adversary’s goal is to get access to information that is not supposed to be exposed. Such information can be regarding data, a model, or properties of data.

• Membership Inference Attack: Shokri [11] proposed the fundamental attack used to check whether model’s training dataset includes a particular data instance, or not known as membership inference attack. Finding out that a record was used to train a specific model while the adversary is fully aware of it is a sign that information is leaking through the model. In certain situations, it may result in a direct violation of privacy. For instance, it can be determined that a patient has a condition if it is known that the patient’s clinical record was used to train a model linked to that disease [12]. Lastly, the viewpoint of the data owner offers another perspective on these kinds of attacks. In this case, the data owner might be able to examine models to find out if the data access permission has been given or not.

• Attribute Inference Attack: Users must answer questions about their age, gender, and other personal details while creating an account to help with user profiling. If hostile third parties manage to steal sensitive user data, the results could be disastrous. However, new studies show that it is highly possible to infer users’ unpublished private information from their interaction history. Age, gender, political affiliation, health, financial position, and other personal details are examples of this type of information. Furthermore, by leveraging the inferred features, the attackers can connect user profiles across platforms and jeopardize anonymity [13]. In such an attribute inference attack [14], the hostile attackers may include advertising, data brokers, or cybercriminals. One example [15] uses the publicly available IMDB user profiles to deanonymize Netflix users successfully.

• Property Inference Attacks: The process of deducing dataset attributes that are neither explicitly stored as features nor directly related to the primary learning objective is known as property inference. For instance, extracting data from medical datasets regarding the ratio of boys to girls. An attacker with access to a trained ML model and an attempt to retrieve some in-depth statistics from the training data is the focus of property inference attacks [16]. Any ML model creator is ready to share model prediction, but no other properties like the environment where data is created or part of the data coming from a specific class [17]. Ateniese et al. [18], first invented property inference attack by doing experiments with Hidden Markov Model and Support Vector Machine.

• Model Extraction Attack: Attacks using model extraction are thought to be a leading cause of vulnerability in ML. In these attacks, an intruder trains a duplicate model that can replicate the original model’s output by repeatedly querying it [19]. Later, attacks of this nature could be a start to more violent ones, including membership inference [20] or adversarial attacks [21]. In addition to developing surrogate models, a few approaches concentrate on extracting knowledge from the final model, like learning parameters in the objective function [22] or details about different aspects of the neural network architecture, like the activation types, optimization algorithm, and number of layers [23].

Privacy-preserving methods to secure privacy

b.

To mitigate the risks and protect privacy, various techniques such as data anonymization and pseudonymization can be employed. In addition to this, some specific methods like differential privacy (DP), local differential privacy (LDP), and privacy-preserving SDG can help us to safeguard sensitive data from inference attacks.

• DP: When an adversary tries to infer information about an individual within a dataset, DP can be a solution. It uses mathematical mechanisms, such as Laplace, Gaussian probability distribution, or exponential mechanism, to add random noise in a dataset. It ensures that the adversary cannot identify individual information by comparing the dataset with other datasets. A trade-off between model utility or accuracy and privacy protection is offered by DP. According to the findings of an assessment of ML models that are differentially private against attacks using membership inference, privacy could only be protected if the models’ effectiveness was greatly diminished [24]. If the added noise is reduced, it increases the privacy leakage.

• LDP: Traditional DP depends on third-party service providers to create noisy data to maintain privacy and assumes that they are trusted. However, even well-established companies are unable to guarantee their customer’ privacy. For instance, in 2018, leakage of Google + social network users’ data by Google, and 52.5 million user accounts were exposed due to bugs in the Google + API in the same year [25]. This raised questions about trust in the third party to manage user privacy. As a solution, LDP [26] can be used by enabling users to make local changes to their own data on their own computers, with just the modified data being uploaded to the server. Despite its potential, LDP has a limited application scope as data is altered locally independent of other users, and lowers utility as the noise included is very large with an increase in complexity.

• Privacy Preserving SDG: Synthetic data refers to artificial data that resembles original data [27]. In case of data scarcity, poor quality data, or improving ML model performance, synthetic data can be used. Along with preserving quality, privacy should be guaranteed when working with sensitive data, such as credit information or medical records. Generation of synthetic data revolves around these two issues. Rather than collection, as it is generated, it does not reveal any private information when privacy constraints are applied. It is important to properly generate synthetic data by adhering to the distribution and patterns of the original data.

SDG workflow

a.

Figure 1 represents steps involved in SDG, such as i) data collection, ii) preprocessing, iii) modelling, and iv) evaluation. The very first step is real data collection, ensuring the privacy of sensitive information by following the data regulations and laws. The second step is about data transformation to remove noise and get clean data that is suitable for modelling. This involves identifying and removing missing values, outliers, data normalization, and feature transformation to get good-quality data. The third step involves the generation of synthetic data by maintaining important properties and characteristics of the original data. This includes the development of SDG models, such as statistical models, ML, and deep learning models. The final step is to assess the generated data to check its quality in terms of utility, fidelity, and privacy.

Figure 1:

Figure 2 depicts the increased interest in the study of fake data generation from 2015 to 2025. The figure shows the total number of research publications indexed in the Scopus database annually from 2015 to 2025. It demonstrates a notable rise in the number of publications throughout time.

Figure 2:

Tabular fake data generation methods

b.

In this part, we attempt to concentrate on previous studies in tabular artificial data generation across sensitive domains, such as healthcare and finance. In addition to tabular SDG, the study covers the metrics for assessing the quality of generated data in terms of utility and fidelity. The study highlights how advanced models, such as GAN, VAE, and large language model, address the challenge of data privacy, data scarcity, and data imbalance.

• Programming Rule-Based Approach: To produce data that closely mimics the characteristics and patterns of real-world data, predefined rules, logical constructions, and simulation models are applied during the programmed generation process. These techniques are pretty flexible and can be modified to replicate the same structures, relationships, and patterns found in actual data. The type of programming approach used in practice is a rule-based approach [29]. Establishing rules that reflect the characteristics and connections of the data, putting these rules into code, and running the program to generate synthetic data are the steps involved in rule-based systems. These systems are highly predictable and reproducible because they are deterministic, meaning that the same rules always produce the same outcomes. If links and constraints are clearly specified, this approach is helpful in producing structured data.

• Statistical Approach: This method creates synthetic data using statistical or probabilistic techniques based on relationships between variables in actual data and the statistical distribution of data points. Statistical methods based on previous studies include Gaussian copula, Bayesian network, bootstrapping, and multivariate normal distribution (MVND). These methods have been used for the augmentation of data, data transformation, and large-scale data generation. The Gaussian copula method [30] is used to handle missing values in various imputation techniques. It models the dependence on data attributes to build the joint distribution, which can be later used for fake data generation.

Standard methods

a.

Prior to the emergence of deep learning-based approaches, several conventional techniques were employed for SDG. This section begins with random oversampling (ROS), a simple way to rectify the imbalance of classes by reproducing minority class samples. We then explored the synthetic minority oversampling technique (SMOTE) and its numerous variants, which generate synthetic samples by interpolating existing minority instances. Then, we examined cluster-based oversampling methods, which leverage clustering algorithms to guide the oversampling process by preserving the underlying data distribution. Finally, we analyzed Gaussian mixture models (GMMs), a probabilistic framework that models data distribution as a combination of multiple Gaussian components, enabling the generation of realistic synthetic samples.

• ROS: ROS duplicates the minority class instances to add a few observations to the dataset. It does not change the data distribution but instead learns the patterns in data with incorrect distribution. It just replicates the samples rather than generating new ones, so a more advanced method, SMOTE, has been developed. random undersampling (RUS) is a method of removing random instances of majority classes to avoid underrepresentation of minority classes. [31] compared both ROS and RUS and proved that ROS performs better than RUS, as it does not affect the majority class instances by increasing minority class samples.

• SMOTE [32]: It is an advanced oversampling method that generates synthetic minority samples rather than simply duplicating existing instances, as done in ROS. The algorithm operates by selecting a minority class sample and identifying the k-nearest neighbors within the same class. For each selected instance, a synthetic sample is created by computing the difference between the feature vector of the instance and that of a randomly chosen neighbor. A random scalar in the range [0,1] is multiplied by this difference, and the result is added to the original instance. As a result, the synthetic sample lies along the line segment connecting the two points in feature space, as shown in Figure 3. This process is iterated until the desired level of oversampling is achieved, helping to create a more continuous and generalized decision boundary for the minority class.

Figure 3:

Despite having more advantages than ROS, SMOTE suffers from a few disadvantages [33], such as ignoring the majority class by focusing on minority class instances, disturbing the original data distribution, because of which variants of SMOTE were developed with some improvement, as shown in Figure 4.

• i) Borderline-SMOTE: Han et al. [34] introduced minority oversampling methods named borderline-SMOTE1 and borderline-SMOTE2, in which oversampling of minority samples only near the borderline was done. The authors proved the improvement of borderline SMOTE over simple SMOTE in terms of F-value and true positive rate. The problem is that it only improved the performance of the worst-performing classifier but did not improve the overall classification performance [33].

• ii) Safe-Level-SMOTE: Bunkhumpornpat et al. [35] presented a technique, Safe-Level-SMOTE, as a solution for the class imbalance problem, which carefully over-samples the data. Unlike SMOTE or Borderline-SMOTE, which may generate synthetic instances in noisy or overlapping regions, this approach ensures that all synthetic samples are generated within safe regions. Each synthetic instance is created near the “safe level,” defined by the number of positive instances among its K-nearest neighbors. By focusing generation in these high-confidence areas, the method inherently avoids noisy boundaries and reduces the risk of introducing ambiguity. As a result, it enhances the quality of synthetic data and leads to more robust classifier performance compared to simple SMOTE or Borderline SMOTE.

• iii) Adaptive Synthetic Sampling (ADASYN): ADASYN [36] is an adaptive synthetic sampling approach that improves imbalanced dataset learning in two ways: i) reducing the bias due to class imbalance and ii) adaptive shifting of classifiers’ decision boundary towards the minority class, which is harder to learn. Since minority class instances with fewer neighboring minority samples are generally more challenging for classifiers to learn, the method applies a weighted sampling strategy that reflects the learning difficulty of each instance. Specifically, it allocates higher weights to harder-to-learn samples with fewer same-class neighbors and lower weights to easier instances. As a result, complicated minority examples generate more synthetic samples, whereas simpler instances contribute less to the creation of synthetic data.

  Like SMOTE, ADASYN creates fake samples along the path segment that links examples of the minority class. ADASYN uses a local density distribution to adaptively determine how many synthetic samples to create for every minority scenario. This adaptive mechanism ensures that more synthetic data is generated for harder-to-learn instances—those located in sparsely populated regions—while fewer samples are created for well-represented examples. As a result, the augmented dataset more accurately reflects the underlying distribution and encourages the classifier to focus on the more challenging decision regions.

• iv) K-means SMOTE: Douzas et al. [37] proposed a new technique to handle class imbalance by combining SMOTE with the K-means clustering algorithm. Most of the old SMOTE techniques were complex and noise-generating. K-Means SMOTE is a three-phase oversampling technique that incorporates clustering into the sample generation process to address the imbalance in class. First, the K-Means algorithm is applied to partition the dataset into clusters. In the second phase, a filtering mechanism is employed to discard clusters containing a low proportion of minority class instances, as these clusters are less informative for synthetic sample generation. Furthermore, the density of minority class samples within each retained cluster informs the allocation of synthetic instances: clusters exhibiting lower minority class density are assigned a greater number of synthetic samples to address class imbalance more effectively. Subsequently, the SMOTE algorithm is applied within each selected cluster to generate synthetic minority instances, leveraging the local data distribution to preserve underlying structure. Experiments proved that the K-means SMOTE handles class imbalance between and within classes more effectively without the generation of noise. The algorithm gives better results on geometric mean, recall, and F1-score.

• Cluster-based oversampling: Jo [38] addressed the issue of performance degradation in classifiers not only because of class imbalance but also resulting in small disjuncts due to imbalanced data. Experiments proved that the classifier gave superior performance to solutions to the class imbalance problem. First, clusters of each class were made separately, and then ROS was performed cluster by cluster. This technique rectifies two imbalances simultaneously: one is between-class imbalance, and the other is within-class imbalance. All clusters, with the exception of the largest, are randomly oversampled for the majority class clusters until their total number of observations equals that of the cluster with the greatest number of data points [39]. The minority class clusters are then randomly oversampled until each cluster includes m/N samples, assuming m as the number of instances of the majority class and N as the number of minority class clusters. This oversampling is the same as that of K-means SMOTE, with differences in a few aspects: K-means SMOTE relies exclusively on the K-means clustering technique and applies clustering across the entire dataset without separating classes. In contrast, cluster-based oversampling offers greater flexibility by allowing the user to choose any clustering algorithm and perform clustering separately for each class. Additionally, K-means SMOTE employs the SMOTE algorithm for generating synthetic samples, whereas cluster-based oversampling uses ROS to balance the class distribution. All the methods except ADASYN neglect the original data distribution, which is highly difficult to estimate.

• GMMs: GMMs are probabilistic models. They assume the data, with its own mean and covariance, is derived from a finite number of Gaussian distributions. The model estimates both the parameters of these Gaussian components and the mixing weights, enabling it to capture complex, multimodal data distributions [40]. The objective of GMMs is to estimate the parameters that define both the mixing proportions and the underlying Gaussian components from the observed data. This estimation is typically performed using the Expectation-Maximization (EM) algorithm, a widely used iterative method. The EM approach alternates between the Maximization (M) phase, which modifies the model parameters to maximize the expected log-likelihood of the observed data, and the Expectation (E) step, which calculates the posterior probability of the latent variables given the current parameter values. Through this iterative process, the EM algorithm converges to a set of parameters that locally maximize the likelihood of the data under the GMM. GMMs are generative in nature and can create synthetic data by sampling from the learned distribution. To make sure that the new data closely resembles the real data, synthetic data points are created from the learnt distribution by randomly choosing a component based on the mixing weights and sampling from the matching Gaussian distribution. The only drawback is that GMM always considers a mixture of Gaussians to represent the correct data, which is not always true. A comparative analysis of the described standard and statistical methods, along with deep learning methods, is given in Table 1.

Figure 4:

Deep learning methods

b.

Deep learning methods are used while generating new data instances, so the category is named as deep learning methods. These are difficult to interpret and understand as they are complex methods. Autoencoders (AE) and GAN are two classes of generative models. They have many variants, but it is not possible to cover all of them, so only a comprehensive overview of the techniques is covered in the literature.

AE

b.i.

AE [41] are feedforward neural networks. The encoder and decoder are two parts of this neural network. The encoder compresses high-dimensional data to low dimensions, while the decoder decodes compressed data into its original form, as shown in Figure 5. An AE not only learns the input data, but it also copies the input samples approximately and generates a copy of the data that resembles the original data. The problem with AE is that there is no standard definition regarding data point distribution in latent space. So, we cannot determine whether the generated data is possible original data. Also, data points within the same class may have significant gaps in latent pace, leading to poor data generation. VAEs were developed as a solution to these AE problems.

Figure 5:

VAE was first proposed by [7], which is an improved version of an AE with a few changes in the loss function and the encoder. A continuous, probabilistic representation of the latent space is encoded by VAEs.

This makes it possible for a VAE to employ variational inference to produce new data instances that closely resemble the original input data, in addition to precisely reconstructing the original input. For each latent variable in training data, VAE builds vectors named the mean vector and the vector of standard deviation. These vectors represent a range of possibilities for a given attribute with its expected variance. By taking this random sampling into consideration, VAE synthesizes new data that resembles the original data, which will be within the range of possibilities.

VAE uses reconstruction loss, such as mean-squared error or cross-entropy loss, to measure the difference between the original input and the output from the decoder. For more accurate reconstruction in latent space, some gradient descent optimization over the parameters of the encoder network will be performed, and the reconstruction error will be reduced. However, along with reconstruction of the original input, VAE targets a new sample generation for which one additional regularization term called Kullback–Leibler divergence (KL divergence) was incorporated. Adding KL divergence helps in reducing the gap between data points in the latent space and ultimately helps in generating a better framework for SDG. Islam et al. [42] used VAE to generate accident data. The author did experiments to compare VAE with SMOTE and ADASYN, and the results proved that VAE performs better than both and gives good classification metrics.

GAN

b.ii.

Goodfellow et al. [6] proposed a GAN architecture for the first time in 2014, which has evolved till now and is capable of writing, painting, and composing. We will see GAN in terms of tabular data generation in this study, as it has many flavors intended for every work.

Generator G and discriminator D are two main functioning models of GANs, as shown in Figure 6. Generator (G) is used to produce synthetic samples that approximate the actual data distribution, generating outputs that are similar statistically but not identical to the real data, thereby avoiding mere replication of training examples. In contrast, discriminator D is tasked with distinguishing between real samples drawn from the actual dataset and fake samples produced by G. Through this adversarial process, both models improve over time: G becomes better at generating realistic data, while D becomes more accurate at classification. Although neural networks are commonly used to implement both G and D due to their expressive power, the framework is not limited to neural architectures; in principle, any differentiable function approximator can be employed.

Figure 6:

Training for the discriminator and training for the generator were conducted independently. Discriminator is trained with original data points and data points generated from the generator G. It gives a probability from 0 to 1 and indicates whether the data belongs to original data or synthetic data. While the generator G is given random noise as input, called the latent space, from a MVND. The output from G has the same features as the input, but no verification exists there. So, the generator is provided with a loss function value, usually binary cross-entropy, which is the difference between the actual value and the computed probability.

Given the output from the discriminator, i.e., the loss function as input to the generator, it tries to improve itself by generating examples resembling the real data and ultimately tries to fool the discriminator. Along with this, discriminators also improve the identification of real and fake data.

Main drawbacks: GAN also carries some disadvantages, which are neither domain-specific nor dependent on any GAN architecture. A few of these problems include the hyperparameter tuning problem, the vanishing gradient problem, the mode collapse, and the uninformative loss. GANs have a significant number of hyperparameters that can be tuned to improve their performance. However, it results in a significant amount of time being required for training and high resource utilization.

The vanishing gradient problem occurs when the gradients used to update weights and generator training become too small. It happens when the discriminator becomes too good at differentiating between original and generated data. This leads to failure of generator training.

Mode collapse is a common issue encountered in GANs, where the generator produces a limited variety of outputs, often generating samples that are very similar to each other. Instead of capturing the full diversity of data distribution, the generator focuses on a few modes, effectively ignoring others. As a result, it fails to generate samples that represent the broader data space. This limited diversity can confuse the discriminator, as it repeatedly sees only a narrow subset of generated samples, making it less effective at distinguishing between real and synthetic data.

Training of generators is difficult to monitor because there is no correlation between the quality of the generated output and the loss. The generator loss is typically referred to as binary cross-entropy, and it does not provide any kind of information. These are a few disadvantages encountered in GAN data generation.

TGAN

a.

Xu and Veeramachaneni [43] proposed TGAN for the first time in 2018 to generate tabular data with categorical and numeric values. Adding the power of deep neural networks, TGAN can generate high-quality tabular synthetic data. The generator and discriminator are the two main parts of TGAN, and they work together within a GAN framework.

A generator based on an long short-term memory network, which is excellent at producing sequential data, serves as the foundation for the generator. It can handle multimodal distributions since it produces both the cluster vector and the value scalar for continuous variables. It ensures realistic one-hot encoded outputs for categorical variables by generating a probability distribution over potential categories.

A multilayer perceptron (MLP) serves as the discriminator, separating synthetic data from actual data. To improve training and avoid mode collapse, it makes use of strategies, such as diversity metrics and minibatch discrimination. The working architecture of TGAN is shown in Figure 7.

Figure 7:

In the study by Xu et al. [43], the authors evaluated TGAN, and experiments proved that TGAN is successful in capturing the correlation between two variables. Also, evaluation in terms of F1 score and accuracy proved that TGAN performs well when machine models are trained with real and synthetic data.

CTGAN

b.

As an improvement in TGAN, Xu et al. [44] proposed CTGAN in 2019. Like TGAN, CTGAN was explicitly designed to synthesize mixed (numeric and discrete) data. Along with preserving the correlation between two columns, it also preserves the joint distribution between all columns. Continuous data features in the dataset often follow a non-Gaussian multimodal distribution. To capture this, CTGAN uses mode-specific normalization. Instead of GMM, it uses a Variational Gaussian Mixture model, which represents continuous feature value as a one-hot encoding vector to represent its sample mode and one scalar that represents the value is normalized according to the mode.

In case of synthesizing categorical data, CTGAN faces two main challenges: one is the sparsity of the one-hot encoded vector, and the second is the imbalancing associated with categorical features. The author employed a conditional generator, which creates synthetic rows that rely on any of the categorical columns, to address imbalanced categorical columns. Also, a sample-by-sample training technique was introduced to examine every discrete value in the dataset uniformly. The CTGAN model is shown in Figure 8.

Figure 8:

To evaluate the performance of CTGAN, the author used a likelihood fitness metric for simulated data evaluation. To evaluate real data performance, ML efficacy is used, as we are unaware of the distribution of the data. In the case of real data, CTGAN outperformed other generative models, such as TableGAN and Medical Generative Adversarial Network (MedGAN). However, in the case of simulated data, it performed relatively better than other models in terms of the likelihood fitness metric. Finally, the author performed an ablation study to evaluate the utility of the conditional generator, mode-specific normalization, and sample-by-sample training. The results showed that they are a very important factor in generating high-quality tabular data.

TabFairGAN

c.

To generate tabular data, Rajabi and Garibay [45] introduced TabFairGAN. The model contains two stages: first, the TabFairGAN model was trained to produce data that resembled the original data. The value function was modified in the second phase to include fairness restrictions, allowing the network to keep learning to produce data that is accurate and fair. As the tabular dataset contains both numerical and categorical variables, categorical values were represented by a one-hot vector. Numerical columns sampled from multimodal distributions were transformed by the author using a quantile transformation method.

The architecture of the TabFairGAN model is shown in Figure 9. The author designed a network as Wasserstein GAN (WGAN) with a gradient penalty, where the generator takes a latent variable as input from a MVND and generates sample data. A critic in a model receives that sample data as input. The generator may produce synthetic data that looks like actual data after both the critic and the generator are properly educated. A fully connected first layer with the Rectified Linear Unit (ReLu) activation function is present in the generator. Several vectors were concatenated to form a hidden layer that generates data like the original data. A fully connected layer with ReLu activation function was implemented for numerical variables, while multiple fully connected layers with Gumbel SoftMax activation were used for discrete variables to generate a one-hot vector. Resulting nodes are then concatenated to feed them as an input to the critic network. Critic is a fully connected layer network with LeakyReLU activation function. Author [45] proved through experiments that TabFairGAN is more effective than TGAN and CTGAN on ML models, MLP, logistic regression, and decision trees in terms of accuracy and F1 score. So, it is quite an effective way to generate tabular data like original data. As shown in Table 1, all the methods used for SDG differ not only in terms of mechanism but also, they differ in privacy aspects and evaluation metrics.

Figure 9: