Figure 1.
Unidirectional context build representation incrementally.
Figure 2.
Bidirectional context words can “see themselves”.
Figure 3.
BERT model input diagram.
Figure 4.
Model architecture of BERT-based Q&A system.
TABLE I.
Number of samples in the data set
| Data Set Number | Dataset Name | ||
|---|---|---|---|
| training set | validation set | test set | |
| 67700 | 47750 | 9975 | 9975 |
TABLE II.
Experimental environment configuration parameters
| Experimental Environment | Configure |
|---|---|
| operating system | Ubuntu |
| development language | Python3.8.8 |
| development framework | Pytorch1.8.0 |
| CPU | Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz2.21 GHz |
| GPU | NVIDIA GeForce GTX3070Ti 8G |
| random access memory (RAM) | Kingston 2400Mhz 16.0 GB |
TABLE III.
Experimental environment configuration parameters
| Experimental Parameters | Retrieve A Value |
|---|---|
| Learning rate | 2e-5 |
| Batch Size | 16 |
| Num of epoch | 20 |
| Length of Maxseq | 128 |
TABLE IV.
Comparative effects of different baseline models
| Modelling | Evaluation Metrics | ||
|---|---|---|---|
| P/% | R/% | F1/% | |
| LSTM | 72.72 | 68.63 | 70.61 |
| Text-CNN | 73.40 | 70.23 | 71.78 |
| BERT | 80.5 | 80.96 | 81.65 |
Figure 5.
Effect of BERT with increasing model size.
Figure 6.
Graphical interface of the construction site Q&A system.