Abstract
Purpose.
This study aims to evaluate the effectiveness of using ChatGPT for a practically important but under-researched task in the scientific literature: the preliminary search and identification of potential business partners or counterparties.
Design/Methodology/Approach.
An experiment was conducted where ChatGPT was tasked with suggesting ten digital marketing agencies for each of three countries - Germany, Latvia, and Poland, including links to these agencies’ homepages. The accuracy of the responses was verified. The study also compared the accuracy of responses between the latest version of ChatGPT (ChatGPT 4o) and the previous version (ChatGPT 4) connected to the Internet.
Findings.
Both versions of ChatGPT were found to make a significant number of errors or inaccuracies when identifying potential business partners that met predefined criteria. Moreover, the frequency of these errors increased when searching in countries with smaller populations and economies. It was also found that the newer version, ChatGPT 4o, did not demonstrate improved accuracy compared to ChatGPT 4, which was connected to the Internet. Both versions of ChatGPT examined in the study provide a higher level of accuracy in identifying potential partners in countries with larger economies and populations.
Originality / Value / Practical Implications.
This research contributes to a practical understanding of the limitations and capabilities of AI tools in identifying business partners, providing valuable information for both practitioners and researchers in the field. This study also provides a benchmark for determining the accuracy of ChatGPT in performing common and important business tasks such as preliminary search and identification of business partners. The results of the study provide a basis for future research to track the progress of language models in similar business applications. The proposed evaluation methodology can be applied to future research aimed at assessing the capabilities of language models in solving business problems.