Leveraging Large Language Models for Aspect-Based Sentiment Analysis: A Restaurant Recommendation System for Entrepreneurs in Lisbon

The Ascendance of Data-Driven Management in Hospitality

2.1

Historically, business decisions, including those in the restaurant sector, often relied heavily on managerial intuition and market perception. However, technological evolution has led in an era where data-driven insights provide concrete support for strategic decision-making. Machine learning, a subset of artificial intelligence, has emerged as a powerful tool that complements human intuition by enabling analysis of vast volumes of data, surpassing the limitations of manual processing (Kozyrkov, 2018). While technology facilitates efficient data processing, human expertise remains crucial for selecting relevant data sources, interpreting results within the business context, and communicating findings effectively (Kozyrkov, 2018). Experts such as Siegel (2018) propose structured processes for implementing predictive models, emphasising collaboration between human analysts and machine capabilities, from defining objectives to operational deployment. This humanmachine symbiosis is central to leveraging data science for effective management, though the increasing autonomy of AI suggests a future point where machines might play a more integrated role in governance (Schwab, 2017). Currently, however, understanding how machines process data is fundamental for utilising these paradigms effectively (Pinski et al., 2024).

Enabling Technologies: Machine Learning, NLP, and LLMs

2.2

Machine Learning (ML) encompasses computational techniques for extracting knowledge from large datasets, situated at the intersection of statistics, artificial intelligence, and computer science (Muller & Guido, 2016). It forms the foundation for more advanced AI applications, requiring human intervention initially to define objectives and finally to interpret results, highlighting a cooperative process (Santos, 2023). Building upon ML, Natural Language Processing (NLP) focuses on enabling computers to understand and process human language. Originating from early machine translation efforts, NLP has evolved significantly, particularly with the integration of ML techniques, enhancing its capacity for tasks like text analysis (Nadkarni et al., 2011). This capability is crucial for analysing customer sentiment expressed in reviews.

The latest advancement in this domain is the Large Language Models (LLMs), AI systems designed for direct interaction using human language. Trained on massive text datasets using deep learning techniques, LLMs like OpenAI’s ChatGPT can engage in fluid conversations, interpret context, identify nuances like sarcasm, and even generate creative text (Törnberg, 2023; Huang et al., 2022). LLMs represent a significant step towards simulating human cognitive capabilities in language understanding and generation (Oliveira, 2019). Their potential to automate analytical tasks previously reliant on human interpretation is a key value proposition (Santos, 2023). The rapid adoption of generative AI across industries underscores its perceived value, with significant organisational investment expected (Chui et al., 2023). However, their use requires careful supervision to mitigate potential biases and ensure the quality of outputs (Rodrigues & Rodrigues, 2023).

Contextualising Restaurant Investment Research with Online Data

2.3

Recent academic literature demonstrates a growing interest in using digital data to inform restaurant location and management strategies. Studies frequently leverage data extracted from popular platforms such as Yelp.com, Koubei.com, Zomato (now DIG-IN in Portugal), and Facebook.com (Shihab & Oishi, 2018; Kim et al., 2022; Wang & Yan, 2017; Lin et al., 2016). These platforms aggregate vast amounts of user-generated reviews and restaurant information, providing rich datasets for analysis. Research spans diverse geographical locations, including major cities in the US (Wang et al., 2016), China (Wang & Yan, 2017), India (Sushmitha, 2020), Bangladesh (Ahmed et al., 2021), Singapore (Lin et al., 2016), and Portugal (Branco et al., 2024). Various analytical techniques have been employed, ranging from traditional ML algorithms like Support Vector Machines (SVM), Decision Trees, and various regression models to more advanced deep learning approaches like Deep Neural Networks (DNN) and transformer models such as BERT and RoBERTa for NLP tasks (Anmoldeep et al., 2020; Shihab & Oishi, 2018; Lin et al., 2016; Wang et al., 2016; Branco et al., 2024). Notably, the literature reviewed prior to this study indicated limited use of the latest generation of LLMs (such as GPT models) for these specific tasks, suggesting an opportunity for investigation (Törnberg, 2023).

Sentiment Analysis and Aspect-Based Sentiment Analysis (ABSA)

2.4

The proliferation of online review platforms has amplified the phenomenon of electronic word-of-mouth (eWOM), significantly impacting restaurant positioning and customer acquisition (Camilo, 2021; Kim et al., 2022). Sentiment analysis is a key NLP task focused on identifying the affective orientation (positive, negative, neutral) within text (Hua et al., 2023). While general sentiment analysis classifies the overall tone of a review, customer feedback is often multifaceted, expressing different sentiments towards various aspects of the experience within a single comment. For instance, a reviewer might praise the location (“lovely”) but criticise the service (“a little slow”), making a single overall sentiment score insufficient for deep understanding. Aspect-Based Sentiment Analysis (ABSA) addresses this limitation by identifying sentiments associated with specific attributes or aspects mentioned in the text (Ahmed et al., 2023; Hua et al., 2024). This technique allows for a more granular analysis, decomposing reviews into specific components (e.g., food, service, ambience, price) and determining the sentiment polarity for each. ABSA provides a richer, more detailed understanding of customer perceptions, highlighting specific areas for improvement or reinforcement (Simmering & Huoviala, 2023). Given the recent advances in LLM capabilities for understanding context and nuance, they present a promising approach to performing ABSA on restaurant reviews.

Service Quality Models and Adaptation for Online Reviews

2.5

Measuring customer satisfaction is complex, involving subjective judgments about service experiences (Uslu & Eren, 2020). Academic research has sought to develop consistent models for this purpose. Early work by Parasuraman et al. (1988) introduced the GAP model, identifying discrepancies between customer expectations and service delivery. Subsequently, the SERVQUAL model proposed five core dimensions for service quality assessment across industries: Tangibility, Reliability, Responsiveness, Assurance, and Empathy (Parasuraman et al., 1988). Recognising the specific context of the restaurant industry, Stevens et al. (1995) adapted SERVQUAL to create DINESERV. This model retains the five core dimensions but refines the associated attributes (originally 29 items) to better reflect the dining experience. The dimensions include physical facilities and appearance (Tangibility), ability to perform the promised service dependably (Reliability), willingness to help customers and provide prompt service (Responsiveness), employee knowledge and courtesy, and their ability to inspire trust (Assurance), and caring, individualised attention (Empathy). Table 1 summarises these dimensions and illustrative criteria.

However, DINESERV was originally designed for data collection via detailed questionnaires administered immediately post-experience. Online reviews, in contrast, tend to be more concise and variable, often focusing on overall impressions or specific salient points rather than systematically addressing all 29 DINESERV items. For example, multiple DINESERV items relating to physical aspects (e.g., attractive décor, comfortable seating, cleanliness) might be condensed into a single general comment about the “ambience” or “cleanliness” in an online review. Therefore, for effective analysis using LLMs on online review data, adapting the DINESERV framework is necessary. This involves consolidating the original items into broader, more representative criteria that capture the essence of each dimension while aligning with the language typically used in online comments. This study undertook such an adaptation, drawing on the DINESERV dimensions and related literature on key restaurant choice attributes (e.g., Choi et al., 2009; Haghighi, 2012; Alves, 2017; Silva et al., 2009; Alhelalat et al., 2017). The goal was to create a synthesized set of attributes linked to the core DINESERV dimensions, suitable for guiding the LLM-based ABSA process. This adapted framework (to be presented in section 3.4) provides a structured yet flexible approach to analysing sentiment expressed in concise online reviews, forming the basis for the methodology employed in this research.

Data Source and Acquisition

3.1

The primary data for this research were provided by ZMTEUROPE, LDA, the company operating the DIG-IN digital restaurant discovery platform (previously Zomato Portugal). Data were obtained through the company’s Datalab initiative. The dataset encompasses two main components pertinent to this study: (1) information on restaurants located within the AML, including characteristics like cuisine type, location, and pricing, and (2) user-generated content associated with these restaurants, specifically numerical ratings and textual reviews (comments). This collaboration provided access to a substantial and relevant dataset reflecting real-world customer experiences within the target geographical area.

Data Characteristics and Pre-processing

3.2

The initial dataset contained information on 15,892 restaurants within the AML and 1,043,806 associated user ratings, of which 474,470 included written comments. Given the large volume and potential for inconsistencies (e.g., null values, inconsistent geographical definitions, and varied cuisine classifications), data pre-processing was performed using the Python programming language and associated data analysis libraries. This phase involved cleaning the data, ensuring correct geographical delimitation of the AML, and standardising restaurant information. Two primary datasets resulted from this process:

– A restaurant dataset containing unique identifiers (restaurant_id), name, cuisine type(s), average rating, price level, location coordinates, operational features (e.g., outdoor seating, takeaway), and review count.

– A reviews dataset linking user comments (original_text) to specific restaurants (restaurant_id) and including the overall rating assigned by the user (general_rating).

Sample Selection for Prototype Development

3.3

Due to the potential diversity and inconsistency in how restaurants self-classify their cuisine types on the platform, an initial analysis identified the ten most frequent cuisine categories within the AML dataset. To ensure a focused yet representative analysis for the prototype development and manage computational costs associated with LLM processing, a specific cuisine category was selected. The ‘Italian’ category was chosen based on three key factors derived from analyzing restaurants offering only one cuisine type: (1) a favorable comment-to-restaurant ratio (average 38.82 comments per restaurant), indicating sufficient opinion density; (2) a substantial number of establishments (115 restaurants), ensuring reasonable representativeness; and (3) a manageable total number of comments (4,464 written comments) that allowed for complete processing within practical resource constraints, unlike categories with significantly higher comment volumes (e.g., ‘Burger’). This selection provided an ideal balance between data richness and processing feasibility for demonstrating the prototype’s capabilities.

Defining a Framework for Sentiment Analysis

3.4

To guide the sentiment analysis, a framework was developed by adapting established service quality models to the specific context of online reviews. Building upon the five dimensions of the DINESERV model (Tangibility, Reliability, Responsiveness, Assurance, Empathy)(Stevens et al., 1995), which are often too detailed for concise online comments, the twenty-nine original items were consolidated into a set of ten attributes frequently discussed in restaurant-selection literature and relevant to customer experience(Choi et al.,2009). An exploratory assessment of the present dataset showed that the average comment contains approximately sixty words, confirming that reviewers rarely address all twenty-nine items systematically and thereby reinforcing the need for aggregation.

The resulting attributes are Food Taste, Food Portion, Service, Price, Ambience / Atmosphere, Food Presentation, Nutritious Food, Restaurant Reputation, Cleanliness and Variety of Healthy Meals (Table 2). This structure preserves the theoretical alignment with the five core dimensions while adopting labels that mirror the vocabulary commonly found in user-generated content.

Then, an LLM (ChatGPT 3.5) was utilised via a specific prompt to assist in mapping these ten attributes to the expanded ten attributes, ensuring that the analysis framework captured the essence of service quality while focusing on aspects commonly mentioned by online reviewers. This adapted framework provides the structure for the aspect-based sentiment analysis (ABSA).

LLM Implementation and Prompt Engineering

3.5

The core sentiment analysis was performed using LLMs accessed via the openrouter.ai platform (https://openrouter.ai), which provides a unified API for interacting with various models.

Model Evaluation

3.6

Several steps were taken to evaluate the accuracy and consistency of the selected LLMs in performing the sentiment analysis task according to the defined framework.

Attribute Keyword Analysis

4.1

The methodological step of identifying keywords associated with the ten defined customer experience attributes (detailed in Section 3.4, Table 2), using LLM analysis (Prompt A, Appendix 1) on a sample of 1,000 comments, revealed distinct frequency patterns. These are summarised in Table 3.

Keywords related to Service (312 mentions), Price (264 mentions), Ambiance/atmosphere (217 mentions), Taste of the food (112 mentions), and Food Portion (119 mentions) were most prevalent. This indicates that customers frequently focus their feedback on these core aspects of the dining experience. Conversely, attributes such as Reputation of the restaurant (19 mentions), Cleanliness (7 mentions), Presentation of the food (45 mentions), and Variety of healthy meals (53 mentions) were mentioned less explicitly. Despite the low frequency for some, all ten attributes were retained for subsequent analysis to ensure comprehensive coverage.

LLM Concordance and Performance Evaluation

4.2

The performance of the selected LLMs (ChatGPT 3.5 Turbo, ChatGPT 4o_2024-05-13, Mistral 7B Instruct) was evaluated through multiple concordance tests as outlined in the methodology (Section 3.6).

Sentiment Analysis of ‘Italian’ Restaurant Reviews

4.3

The entire set of 4,464 comments for the 115 ‘Italian’ restaurants in the AML (selected as per Section 3.3) was processed using the ChatGPT 3.5 Turbo model (guided by Prompt B, Appendix 2) to generate attribute-level sentiment scores. A comparison between the overall sentiment rating derived from the model’s attribute analysis and the customer’s original rating showed a high degree of alignment. Specifically, 83.96% of the comments processed had a difference of 1.0 Likert scale points or less between the customer’s rating and the model’s derived rating. This finding suggests that even the less advanced GPT-3.5 Turbo model demonstrated considerable competence in capturing the overall sentiment expressed across the various attributes mentioned in the reviews.

Prototype Tool Functionality and Insights

4.4

The sentiment analysis results derived from processing the 4,464 comments were integrated into an interactive prototype tool. This tool is designed to provide potential investors with actionable insights through two main visual components:

The first component is an Interactive Map, which visualises a selected geographical area (e.g., Marquês de Pombal, as shown in Figure 1a). This map displays the locations of competing restaurants within the chosen category (‘Italian’) and employs features like a colour grid to indicate market saturation (density of competitors). Users can click on individual restaurant pins to access detailed pop-ups containing basic information and summarised sentiment analysis results (Figure 1b). This interface provides a clear and immediate overview of the competitive landscape in the selected vicinity.

Figure 1:

The second component features Restaurant Performance Radar Charts. For selected competitors, the tool generates radar charts displaying the average sentiment score (derived from LLM analysis of reviews) for each of the ten defined attributes (e.g., Taste, Service, Price, Ambience), as illustrated in Figure 2. This allows investors to quickly visualise and compare the perceived strengths and weaknesses of competitors based on customer feedback, moving beyond simple overall ratings to understand performance on specific aspects crucial to customer experience.

Figure 2:

Together, these features centralise competitive information and sentiment analysis, enabling investors to identify market opportunities and potential areas for differentiation based on detailed, data-driven insights derived directly from customer comments.