Figure 1.
Model structure of the Wide & Deep Learning model.
Figure 2.
Shared-Bottom model.
Figure 3.
FM_Shared Expert Multi-Objective Dependency (FSMD) Mode
Figure 4.
Multi-objective Base Model.
Figure 5.
Multi-gate MoE Model.
Figure 6.
Internal Structure of Gated Network.
Figure 7.
FM_Gate (FG) Model.
Figure 8.
FM_Sharing the deep part of the Expert (FS) Model.
Figure 9.
FM_Shared Expert Multi-Objective Dependency(FSD)Model.
TABLE I.
PARAMETER SETTINGS
| Parameter Name | Value |
|---|---|
| batch_size | 256 |
| optimizer | adam |
| learning_rate | 0.001 |
| embedding_size | 4 |
| dnn_layers | (512, 256) |
| dnn_use_bn | True |
TABLE II.
Results of the UCI Census Income Dataset
| Models | AUC/ Income | AUC/ Marital | Mean |
|---|---|---|---|
| Single-Task | 0.9198 | 0.9748 | 0.9473 |
| Shared-Bottom | 0.9148 | 0.9754 | 0.9451 |
| MMoE | 0.9152 | 0.9756 | 0.9454 |
| PLE | 0.9161 | 0.9764 | 0.9463 |
| Base | 0.9134 | 0.9706 | 0.9420 |
| FG | 0.9146 | 0.9693 | 0.9420 |
| FS | 0.9216 | 0.9756 | 0.9486 |
TABLE III.
Parameter Settings
| Parameter Name | Value |
|---|---|
| batch_size | 2048 |
| optimizer | adam |
| learning_rate | 0.001 |
| embedding_size | 4 |
| dnn_layers | (256, 128) |
| dnn_use_bn | True |
| dropout | 0.5 |
TABLE IV.
Parameter Settings
| Models | AUC/ Income | AUC/ Marital | Mean |
|---|---|---|---|
| Single-Task | 0.7192 | 0.6635 | 0.6914 |
| DeepFM | 0.719 | 0.658 | 0.689 |
| Shared-Bottom | 0.7184 | 0.6916 | 0.705 |
| ESMM | 0.7189 | 0.6897 | 0.7043 |
| MMoE | 0.7201 | 0.7057 | 0.7129 |
| Base | 0.7193 | 0.7086 | 0.714 |
| FG | 0.7201 | 0.7084 | 0.7143 |
| FS | 0.7204 | 0.7114 | 0.7159 |
| FSMD | 0.7205 | 0.7117 | 0.7161 |
Figure 10.
The seesaw phenomenon in each model under complex target association
Figure 11.
AUC comparison of network shapes.
Figure 12.
AUC comparison of the number of layers in a gated network.
Figure 13.
AUC comparison of the number of experts.