Citation:
Nicolescu, L.; Tudorache,
M.T. Human-Computer Interaction in
Customer Service:
The Experience
with AI Chatbots—A Systematic
Literature Review.
Electronics
2022
,
11
, 1579.
https://doi.org/
10.3390/electronics11101579
Academic Editors:
Yanping Zhang,
Zhifeng Xiao and Jianjun Yang
Received:
18 April 2022
Accepted:
13 May 2022
Published:
15 May 2022
Publisher’s Note:
MDPI stays neutral
with regard to jurisdictional claims in
published maps and institutional affil-
iations.
Copyright:
©
2022
by
the
authors.
Licensee
MDPI,
Basel,
Switzerland.
This
article
is
an
open
access
article
distributed
under
the
terms
and
conditions
of
the
Creative
Commons
Attribution
(CC
BY)
license
(https://
creativecommons.org/licenses/by/
4.0/).
electronics
Review
Human-Computer Interaction in Customer Service:
The Experience with AI Chatbots—A Systematic
Literature Review
Luminit
,
a Nicolescu
1,
*
and Monica Teodora Tudorache
2
1
Department of International Business and Economics, Bucharest University of Economic Studies,
Piat
,
a Roman
ă
6, 52632 Bucharest, Romania
2
Wirtschaftsinformatik, Otto Friedrich University of Bamberg, Kapuzinerstraße 16, 96047 Bamberg, Germany;
monitudorache@yahoo.com
*
Correspondence:
luminicolescu@yahoo.com
Abstract:
Artificial
intelligence
(AI)
conversational
agents
(CA)
or
chatbots
represent
one
of
the
technologies
that
can
provide
automated
customer
service
for
companies,
a
trend
encountered
in
recent
years.
Chatbot
use
is
beneficial
for
companies
when
associated
with
positive
customer
experience.
The purpose of this paper is to analyze the overall customer experience with customer
service
chatbots
in
order
to
identify
the
main
influencing
factors
for
customer
experience
with
customer service chatbots and to identify the resulting dimensions of customer experience (such
as
perceptions/attitudes
and
feelings
and
also
responses
and
behaviors).
The
analysis
uses
the
systematic
literature
review
(SLR)
method
and
includes
a
sample
of
40
publications
that
present
empirical
studies.
The
results
illustrate
that
the
main
influencing
factors
of
customer
experience
with chatbots are grouped in three categories:
chatbot-related, customer-related, and context-related
factors, where the chatbot-related factors are further categorized in:
functional features of chatbots,
system
features
of
chatbots
and
anthropomorphic
features
of
chatbots.
The
multitude
of
factors
of customer experience result in either positive or negative perceptions/attitudes and feelings of
customers.
At
the
same
time,
customers
respond
by
manifesting
their
intentions
and/or
their
behaviors towards either the technology itself (chatbot usage continuation and acceptance of chatbot
recommendations) or towards the company (buying and recommending products).
According to
empirical studies, the most influential factors when using chatbots for customer service are response
relevance and problem resolution, which usually result in positive customer satisfaction, increased
probability for chatbots usage continuation, product purchases, and product recommendations.
Keywords:
AI conversational agents; AI chatbots; customer service; customer experience
1.
Introduction
Artificial intelligence (AI) conversational agents (CA), also known as AI chatbots are
seen as software applications that are capable to communicate through natural language [
1
],
and they represent interactive systems in which human-computer interaction takes place.
In the recent years, CA started to be used on a large scale, due to newer developments of
artificial intelligence and machine learning and also the fact that, after 2016, Microsoft and
Facebook launched frameworks for the integration of CA on their platforms [
2
,
3
].
Conversational agents are used in diverse fields and contexts (entertainment,
mar-
keting,
education,
health
care,
support
systems,
culture
diffusion)
[
3
]
as,
at
present,
AI
technologies and machine learning allow AI enabled chatbots to mimic human behavior
and enter conversational situations [
4
]. However, one important area in which CA/chatbots
are used is the customer service activity, as AI enabled chatbots are seen as a promising
technology for service providers [
1
] by providing automated customer service [
4
].
In the
Electronics
2022
,
11
, 1579.
https://doi.org/10.3390/electronics11101579
https://www.mdpi.com/journal/electronics
Electronics
2022
,
11
, 1579
2 of 24
last two years of the COVID-19 pandemic, the development of this specific IT-enabled ser-
vice was prevalent in many companies, the year 2021 being a decisive year for the inclusion
of the AI CA/chatbot technology for customer service activities, as we will present later in
the paper.
In this context, expectations are high in the customer service field when using the AI
enabled CA/chatbot technology [
2
].
The achievement of the benefits that are potentially
associated with the use of CA/chatbots for customer service requires positive user expe-
riences [
5
].
Therefore, detailed knowledge of the customers’ experiences with customer
service chatbots is one field of interest for both practitioners and researchers [
4
].
Practical
motivation
.
Customer
service
is
an
emerging
area
for
the
application
of
chatbots, as AI CA are a means to automate customer service and make this activity more
cost efficient for service providers.
Practitioners [
6
] consider that there will be an increase
in the adoption of chatbots for customer service as AI-enabled virtual agents can work with
most customer relationship management (CRM) activities and allow for CRM automation.
They consider that smart automation is the biggest transformation of contact centers during
2021, as smart automation took over the frontline of customer service across industries.
In
addition, the global chatbot market is expected to grow up to 10.08 million $ by 2026 [
7
].
However, the success of using AI CA for customer service depends on the experiences that
customers have with the automated customer services provided by CA/chatbots.
Service
providers can enhance their AI-enabled customer service activity and improve their CA
interaction design only when they are fully aware of how customers feel like, how they
act, and what are the factors that influence their feelings and behaviors when using CA for
customer service.
Theoretical
motivation
.
The
analysis
of
the
customer
experience
can
be
conducted
based on user experience theories, as customers represent one category of users for AI CA.
User experience refers to how a person perceives and responds to the use or anticipated
use
of
a
product,
system
or
service
[
4
].
Different
studies
look
solely
at
one
or
another
particular aspect of users’ experiences with AI CA/chatbots regarding both perceptions
(trust, enjoyment, satisfaction) and/or responses (continuance, purchase) [
8
,
9
].
The creation
of an overall image of customers’ experiences with AI chatbots can bring clarification on
how this concept applies in relationship to AI conversational agents.
Therefore, one first
aim of this paper is to see what the components of the overall customer experience with
CA are and what the characteristics of the interaction process are.
Secondly, it is of interest
to see how the overall users’ experiences with CA apply to the particular field of customer
service, an activity offered by companies to strengthen customer satisfaction [
10
].
User-
centered evaluations of CA/chatbots are necessary, as there is the need for more knowledge
about CA/chatbot experiences from the perspectives of the end users [
11
,
12
], in the present
case, customers.
To fulfil these aims, a systematic literature review (SLR) is conducted.
Other literature reviews look at human–chatbot interaction from different perspectives:
technical [
13
], historical [
14
] or only one particular perspective of the interaction:
customer
loyalty [
15
].
To our knowledge, there is no literature review to look at the overall customers’
experience
(perceptions/attitudes/feelings
and
responses/behaviors)
with
AI
CA
and
chatbots for customer service from the end user perspective.
Therefore, the present study
tries to fill in this research gap by specifically proposing a systematic literature review to
analyze the overall customer experience with AI CA for customer service.
The research questions that this research tries to answer are the following:
RQ1.: What are the factors that influence the customer experiences with AI CA/chatbots
for customer service?
RQ2.:
What are the resulting dimensions of the overall customer experience with AI
CA/chatbots for customer service?
The dimensions of the overall customer experience refer to two main components that
are considered in the present study:
(a) the perceptions, attitudes, and feelings of customers
when using AI CA/chatbots and (b) the responses and behaviors that customers have after
using AI CA/chatbots.
Electronics
2022
,
11
, 1579
3 of 24
The rest of the paper is structured as follows.
In Section
2
, the theoretical background
necessary to answer the research questions is presented.
Following, Section
3
presents the
materials and methods employed for the present SLR. Section
4
includes the results of SLR
and discussions related to the findings.
At last, Section
5
concludes the paper.
2.
Theoretical Background
2.1.
Conversational Agents—Definition, History and Classifications
Conversational agents or chatbots have been defined in different ways. Table
1
presents
a few definitions.
Table 1.
Conversational agents/chatbots definitions.
Definition
Reference
Conversational agents/chatbots in general
A software which can chat with people by using artificial intelligence
Alam et al.
[
16
] (p.
33)
A computer program that simulates human–human conversation.
Ho et al.
[
17
] (p.
712)
Conversational agents/chatbots for customer service
An artificial intelligent program that can interact with consumers via
different messaging apps.
Riikkinen et al.
[
18
] (p.
1148)
The idea of chatbot as a conversational agent has been developed in 1950’s by Alan
Turing, who was curious to find out if a computer program could talk to people without
them realizing that the speaker is artificial [
3
].
Adamopoulou and Moussiades presented a
short history of chatbots/CA development over time starting with chatbot ELIZA (1966),
continuing
with
PARRY
(1972),
Jabberwacky
(1988),
TINYMUD
(1991),
ALICE
(1995),
SmartChild (2001), Siri (2010), Watson (2011), Google Now (2012), Google Assistant (2016),
Cortana (2014), Alexa (2014), each of them representing a more evolved bot as compared to
the previous ones.
Recently, more advanced technologies started to be used with chatbots
(shifting from pattern-matching to machine learning and AI) [
3
].
Starting in 2016, new AI
advancements allowed companies to develop CA for their brands or services.
CA/chatbots
can
be
classified
according
to
different
criteria
and,
here,
there
are
some
relevant
classifications.
According
to
the
response
mechanisms
used,
there
are
two
major
response
mechanisms
used
by
CA/chatbots:
(a)
the
rule-based
model
also
called the retrieve-based model or template-based model and (b) the generative model.
The rule based/retrieve-based model uses predefined sets of responses that are retrieved
from a large collection and are offered in the conversation.
These are the simplest forms
of
CA/chatbots.
The
generative
model
implies
that
the
CA/chatbot
generates
a
new
response from scratch, and produces completely new sentences based on AI and machine
learning [
19
,
20
].
These are the AI CA. There are also hybrid CA/chatbot systems that have
partly defined and partly free responses [
21
].
Another important classification considers
the knowledge domain of CA/chatbots.
There are:
(a) open-ended domain CA/chatbots
(that have knowledge and can answer questions from any domain) and (b) closed-ended
domain CA/chatbots (that have knowledge and can answer only questions that belong to
a particular domain) [
3
].
According to the type of interaction, there are:
(a) chatbots for customer service (pro-
viding information, help, advice by a company, government or a non-profit organization);
(b) personal assistant chatbots that serve the user continuously (Alexa); (c) content curation
chatbots that offer access to useful information (news, weather) and entertainment, and (d)
chatbots for coaching that have the purpose to guide the user with specific tasks (education
or therapy) [
3
].
Considering these three criteria, the CA/chatbots of interest for the present study are
machine learning-AI chatbots/CA, specialized in a closed domain pertaining to customer
service interaction for businesses.
Even though CA/chatbots exist for a long time, only recently (after 2016), companies
started to use chatbots for communicating with clients and for customer service.
Electronics
2022
,
11
, 1579
4 of 24
A
virtual
customer
service
agent
(a
chatbot,
a
conversational
agent)
consists
of
“computer-generated
characters
that
are
able
to
interact
with
customers
and
simulate
behavior of human company representatives through artificial intelligence” [
22
] (p.
530).
The next section looks at customer service and the use of conversational agents/chatbots
for customer services.
2.2.
Customer Service and Conversational Agents
Customers
are
one
very
important
category
of
stakeholders
for
any
organization,
therefore ensuring their satisfaction is (or should be) one of the main preoccupations for
companies.
One way of creating customer satisfaction is through good customer service.
Customer service has been defined as “the interaction that takes place between somebody
from a company and the customer and links all tasks and functions in a company” [
23
]
(p. 4).
In very recent years, IT-enabled digital systems started to be used by companies for
providing customer service activities with the purpose to increase customer satisfaction.
Among those, one technology that took prevalence is the chatbot technology that includes
AI conversational agents that interact with customers.
Generally
speaking,
the
virtual
conversational
agents
are
used
by
companies
for
fulfilling different tasks related to customer service such as:
solving complaints, identifying
items for purchase, making recommendations [
7
,
24
].
The
purpose
of
using
chatbots
for
customer
service
is
to
encourage
the
positive
development of interaction with customers [
25
] by making use of the chatbots’ benefits.
Researchers
agree
that
the
use
of
the
chatbot
technology
determines
both
benefits
and
challenges
from
the
perspective
of
both
companies
and
consumers.
Among
the
main
benefits for companies are cost reduction,
time saving for customer service tasks [
7
,
24
],
also the possibility to serve multiple customers simultaneously [
3
].
At the same time, for
consumers, benefits refer to 24/7 access to customer service allowing them to post their
questions at any time, therefore increasing customer satisfaction [
3
].
Digital
transformation
is
considered
to
bring
new
ways
for
value
creation
for
cus-
tomers, such as automation, individualization, interaction, and transparency and control,
that further can determine perceived customer benefits, such as convenience, relevance,
experience,
empowerment,
and
savings
[
26
],
benefits
that
can
apply
when
consumers
interact with CA/chatbots, as well.
However, there are also a number of challenges and limitations related to the CA/chatbot
use for both companies and consumers and these include:
risks related to personal data se-
curity; limitations regarding the level of understanding of messages (they do not recognize
the intention of their interlocutor) and the production of natural language, that can create
disappointment for the user and drive the customer away [
3
] (p.
13).
Therefore, the opinions of customers using CA/chatbot become important.
2.3.
Customer Experience and Conversational Agents
As
we
have
seen,
customer
experience
is
essential
for
the
success
of
CA
used
in
customer
service
activities.
As
customers
are
one
important
category
of
IS
users,
the
theories on user experience with IS are used as a starting point for the development of the
theoretical framework developed to analyze and answer the research questions.
User experience, in general, can be defined as “a person’s perceptions and responses
resulting from the use and/or the anticipated use of a product, system or service.
Users’
perceptions and responses include the users’ emotions, beliefs, preferences, perceptions,
comfort,
behaviors,
and accomplishments that occur before,
during and after use” [
27
],
(3.2.3.).
At
the
same
time,
chatbot
user
experience
is
seen
as
“concerning
how
users
perceive and respond to chatbots and how chatbot layout, interaction mechanisms and
conversational content influences perceptions and responses” [
12
] (p.
2924).
Based on the definition, it can be considered that the dimensions of customer experi-
ence with CA include:
(a) the perceptions, attitudes and feelings of customers when using
Electronics
2022
,
11
, 1579
5 of 24
AI CA/chatbots, on the one hand, and (b) the responses and behaviors that customers have
after using AI CA/chatbots, on the other hand.
In order to characterize the customers’ experience with CA/chatbots, there is the need
to identify the factors that influence the customer experience in the interaction with CA,
on the one hand, and the resulting dimensions of customer experience, on the other hand.
Two types of theories are considered, for this purpose:
(a) the IT acceptance models that
help identify influencing factors and (b) the IT user satisfaction models that help identify
results of customer experience with CA/chatbots.
Various IT acceptance models exist in the literature, such as Technology Acceptance
Model (TAM) and the Unified Theory of Adoption and Use of Technology (UTAUT) and
its
extension
[
28
],
and
they
propose
a
variety
of
influencing
factors
for
IS
use
(such
as
usefulness, ease of use, performance expectancy, effort expectancy, social influence, etc.).
The models can also be applied in the context of CA/chatbots for customer service.
Given
the large variety of factors that can influence the customer experience with CA, there is
the need to group these factors.
Different typologies classified factors influencing the use
of IS [
2
,
29
–
31
].
The typology selected to group factors influencing customer experience
with CA in this study is the one proposed by [
8
] (p.
9) for factors assumed to affect trust in
chatbots for customer service, which include chatbot-related factors, environment- related
factors, and user-related factors.
The other category of theoretical models refers to the user satisfaction models and
explain the behavior of consumers.
One such model is the expectation–confirmation theory,
which states that satisfaction with Information Systems (IS) is predicted by user’s confir-
mation of expectation from IS and further on determines the IS continuance intention [
32
]
(p.
366).
Another model is DeLone and McLean’s IS success model, which states that user
satisfaction and the intention to use IS depend on information quality, system quality, and
service quality.
These models are used to identify the dimensions of customer experiences
in the present study [
33
] (p.
24).
The
above-presented
theoretical
concepts
and
models,
namely
the
IT
acceptance
models and factors’ typology, on the one hand, and the IS user satisfaction models and
the definition of user experience, on the other hand, are used to develop the theoretical
framework adopted to analyze customer experiences with AI CA/chatbots.
The theoretical
framework applied in this paper is presented in Figure
1
.
Influencing factors of
CA/chatbot use
Conversational
agent/chatbot
related factors
User related
factors
Context/
environmental
related factors
Customers' perceptions,
attitudes and feelings
related to CA/chatbot
use
Positive
Negative
Customers' responses
and behaviours related
to CA/chatbot use
Towards the
CA/chatbot
Towards the
company
Figure 1.
Theoretical framework for analysis of customer experience with AI CA/chatbot.
Electronics
2022
,
11
, 1579
6 of 24
3.
Materials and Methods
The research method utilized in this study is the systematic literature review that was
characterized as a way to identify, evaluate, and interpret all available research relevant to
a research topic [
34
].
Among the types of SLR, the describing literature review is selected
as it allows to summarize the existing literature and better understand the current state of
knowledge [
35
,
36
] in the field of customers’ experiences with AI CA for customer services.
In order to ensure replicability of the study, Okoli’s [
37
] guidelines for conducting the
literature review are used, a protocol that is tailored for IS research and it is highly relevant
for
the
present
study.
Okoli
[
37
]
proposed
four
phases
with
eight
steps
in
total
for
the
literature review process (see Figure
2
), steps that have been applied for the present research.
The
planning phase
has two steps:
(1) establish the goal and the purpose of the study
and
(2)
establish
a
research
protocol
and
train
researchers
[
37
]
(p.
885).
The
general
purpose of the present paper is to analyze the progress of research on the overall customer
experience when using AI CA for customer service.
The research protocol established for
the present literature review is presented in detail in this section.
The
selection phase
explains how the literature to be reviewed is selected and, according
to Okoli [
37
] (p.
885), it has two steps:
(3) apply initial screening and (4) apply search of the
relevant literature.
In the present research work, the initial screening is based on the online
search of five databases:
EBSCO, Web of Science, Science Direct, ACM Digital Library, and
Google Scholar, which were searched using keywords.
The
search
string
strategy
is
developed
in
strong
connection
with
the
terms
of
the
research questions and also includes synonyms for these terms [
34
]. The Bolean practice was
used and OR and AND operators were included, as well as sign as (*) or (“”), as required
by the use conditions of each database.
The operator OR was included between keywords
considered to be synonyms and the operator AND was used to ensure the inclusion in the
search of all terms simultaneously.
The final search string was “AI conversational agent OR
AI chatbot AND user experience OR customer experience OR customer satisfaction AND
customer service OR customer relationship management OR marketing”.
The key words
were searched in title, abstract, and text (if available).
In order to ensure the quality of the publication sample, specific criteria were applied
starting with the initial screening stage, criteria that also considered the qualitative pro-
cedures used by the journals of the publications.
Therefore, the initial screening applied
the databases’ filters and the inclusion criteria employed at this stage were:
(a) papers that
include the keywords according to the search string; (b) papers published in peer-reviewed
journals and in highly ranked conference proceedings (also needed according to [
38
]) (for
ensuring
quality);
(c)
papers
that
have
access
to
full
text
(to
ensure
access);
(d)
papers
that are published in English (to ensure understanding), and (e) papers published during
2010–March 2022 (to ensure recency and relevance for AI CA/chatbots, but also to enclose
early
publications
on
the
topic).
Duplicates
of
papers
(papers
found
in
more
than
one
database) were removed.
As
a
result
of
the
initial
screening,
186
papers
were
selected.
Table
2
presents
the
results of the keyword search (and associated criteria) for each database.
Electronics
2022
,
11
, 1579
7 of 24
PLANNING
SELECTION
EXTRACTION
PLANNING
1. Goal of SLR
2. Establish Protocol
and Train
Researchers
3. Apply Initial
Screening
4. Apply Search of
Relevant Literature
5. Extract Data
6. Evaluate Quality
7. Synthesize
Findings
8. Write the Review
Figure 2.
The phases and steps of the literature review process—Adapted from [
37
] (p.
43) and [
39
]
(p.
3).
Table 2.
Results of the keywords search by database and other sources.
Database/Source
Initial Hits
(Keywords)
Hits after
Initial Screening
Hits after
Abstract Reading
Hits after
Full Text
Reading
EBSCO
68
11
9
8
Web of Science
84
19
14
11
Science Direct
392
80
13
5
ACM Digital
Library
295
32
12
4
Google Scholar
156
44
15
5
Citation screening
1
-
-
12
5
Additional papers
-
-
-
2
TOTAL
995
186
75
40
1
Backward and forward screening.
These papers were selected further during the next step, the search of literature based
on content-related inclusion/exclusion criteria that are presented in Table
3
.
In order to
ensure
a
high
quality
of
the
publications
to
be
included
in
the
sample,
only
academic
publications were considered (published in peer-reviewed journals and top conferences
Electronics
2022
,
11
, 1579
8 of 24
proceedings), while non-academic publications were excluded.
In the sample, only pub-
lications with content were selected that directly answered to the research questions of
the present paper, while remotely connected subject-wise publications were excluded.
A
more detailed thematic selection was done by including only publications that focused
on AI CA and chatbots used for customer service in the business field, while publications
referring
to
other
types
of
use
of
AI
CA
and
chatbots
(such
as
social
companion
or
for
learning purposes) or publications dealing with the use of chatbots in other sectors of the
economy (public administration, the health sector or education) were excluded.
In order to
be able to analyze the customer experience, only publications that researched AI CA and
chatbots from the customers’ perspectives were included, while the publications that dis-
cussed the topic from the companies’ perspective were excluded.
Finally, methodologically,
only publications presenting empirical studies with clear research methodologies were
included, while reviews of literature were excluded.
The inclusion/exclusion criteria were
first applied to the abstracts of publications and then to the full texts of the publications.
It resulted in 33 publications selected based on keywords.
The search process continued
with the backward (looking at publication references) and forward (looking at publication
citations) search of the most relevant publications [
40
] and five more publications were
included.
In addition, two additional papers were included that were relevant in content,
even though they do not fit into the initial screening criteria [
41
].
Figure
3
presents the
process of the online literature search using the PRISMA 2020 flow diagram [
42
].
Table 3.
Inclusion and exclusion criteria for relevant literature search (publication screening).
Inclusion Criteria
Exclusion Criteria
(a) Answer directly to research questions
(a) Do not answer directly research questions
(b) Only academic publications
(b) Publications that are not academic
(c) Focus on AI CA and chatbots for customer service
(c)Focus on other CA/chatbots related aspects (design)
(d) Include and focus on customers’ perspective
(d) Focus solely on company’s perspective of using AI
CA/chatbots
(e) Only primary studies that include empirical results obtained
based on a specified research methodology
(e) Studies that include reviews of literature
(f) Studies that refer to the use of CA in only business (retailing,
transportation, banking, hospitality)
(f) Studies that refer to the use of CA in non-business sectors
(health, education, public administration)
(g) Studies that refer to the use of CA for other purposes than
business customer service (social companion, robotics, learning)
At the end
of the entire literature selection process,
the
number of publications in-
cluded in the final literature sample is 40.
The
extraction phase
refers to taking information from each paper for synthesizing it and
has two steps:
(5) extract data and (6) evaluate and appraise quality [
37
] (p.
885).
The data
were extracted from the 40 publications based on an extraction form that was developed
using two models [
34
,
43
].
In order to accommodate the interdisciplinary character of the
present study, the extraction form specifically designed for this study used two models
originating from two different fields as starting point:
software engineering [
34
] and social
sciences [
43
].
The
extraction
form
was
developed
to
serve
two
purposes:
take
out
and
organize
the relevant information from each paper, but also to evaluate and appraise the quality of
the paper.
For these two purposes, the extraction form included the paper identification
information (title, journal, authors, origin of authors, year of publication, journal domain,
geographical setting of the study,
industry under investigation,
time of data collection)
and also included detailed information about both methodological considerations and also
results of the study.
Appendix
A
presents the extraction form used in the present study for
both purposes:
information synthesis and quality evaluation of papers.
Electronics
2022
,
11
, 1579
9 of 24
Figure 3.
Results in the stages of the literature selection process (PRISMA diagram flow) [
42
].
It can be stated that the quality of the selected sample publications was ensured from
the early stages of the research, given the rigorous multi-stage selection process they went
through in order to be included in the final sample of literature (initial screening and content
screening, both including extensive and quality-oriented inclusion/exclusion criteria).
At
this stage, the quality of the publications is appraised based on the information obtained
via the extraction form that allowed the researchers to evaluate the quality of the papers
using two criteria:
the methodological thoroughness employed by the studies and also the
level of detail in terms of results of the empirical studies and consequent implications of
the findings.
In the final sample, publications were included that presented in detail all
the required methodological aspects (objectives and research questions, research design,
research methods, sources of data, sample characteristics—participants and sample size,
location,
industry,
data
collection
period,
research
instruments,
and
methods
for
data
analysis).
The existence of a very detailed presentation of the methodological organization
of the empirical research and of the methods of data analysis were used as a dichotomous
criterion to evaluate the quality of the studies and to accept them in the final sample.
From
Electronics
2022
,
11
, 1579
10 of 24
the perspective of the results of the study,
the publications sample included the papers
that described their empirical findings in detail and studies with empirical results related
to our research questions (influencing factors for customer experience with AI chatbots
for customer service and the dimensions of the overall user experience, as defined in the
present study as feelings, attitudes, and perceptions of customers, as well as reactions and
behaviors of customers).
All publications with a thorough description of the methodology employed for the
empirical research passed the qualitative threshold and were included in the sample and
all publications with a detailed description of results, containing topic-relevant results and
also discussing implications of their empirical results, also qualified from qualitative point
of view and were included in the final sample of publications.
The
execution
phase
has
two
steps:
(7)
synthesize
findings
and
analyze
studies
and
(8) write the review [
37
] (p.
885).
The analysis of findings was handled by aggregating,
discussing,
organizing,
and
comparing
the
selected
publications
[
37
].
The
data
were
synthesized using the narrative synthesis that according to Okoli comprises tabulating the
included studies and describing the study sample of publications [
37
].
This is presented in
the next section.
4.
Results and Discussion
The present narrative synthesis includes two types of analyses:
(a) a description of the
publications selected for review and (b) the thematic analysis of the publications according
to the research questions and based on the proposed theoretical framework.
4.1.
Descriptive Analysis—The Organization of the Studies
The descriptive analysis presents the results from the analysis of 40 studies that em-
pirically researched customers’ experiences with customer service conversational agents,
according to:
(a) year of publication, (b) countries of origin of authors (first author) and
countries in which the empirical research was conducted, (c) the subject area of the publi-
cation venues, (d) the research methods and tools adopted, and (e) industries involved in
the studies.
It can be observed that the increased development of AI CA for customer service after
2016 and the recency of the uprise of this technology use for customer service was reflected
also in an increased interest of researchers in this topic.
The majority of the publications on
this topic (77%) were published in the last years 2020–2022, see Figure
4
a.
It is expected
that
the
topic
will
be
further
researched
in
the
future,
as
the
use
of
this
technology
is
foreseen to increase in the business context.
Most of the publications (75%) were authored
by researchers originating from European countries, among which authors from Germany,
Norway, and UK were the most numerous.
Likewise, almost half of the empirical research
was also conducted in European countries, followed by research conducted in the US and
in Asian countries, see Figure
4
b,c.
At present, Europe represents a pole of research on the topic of AI CA/chatbots in
customer service and in customer experience-related subjects.
The
main
venue
of
publication
was
represented
by
journals
from
the
information
systems
and
computing-related
domains
(over
55%),
see
Table
4
.
Another
important
publication venue was represented by journals from the marketing domain, explained by
the specificity of the topic:
customer service (that is a marketing activity) and customer
experience (that is a marketing concern). Other publication venues were journals publishing
papers specific to certain industries that have been studied (tourism, retailing, services).
Electronics
2022
,
11
, 1579
11 of 24
(
a
)
(
b
)
(
c
)
0
0
1
0
0
0
0
0
3
5
11
15
5
0
5
10
15
20
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
28
6
4
1
1
Europe
US
Asia
New Zealand
Latin America
19
7
7
1
6
Europe
US
Asia
Latin America
N/A
Figure 4.
Characteristics of the publications in the sample.
(
a
) Publication year; (
b
) First author origin;
(
c
) Location of empirical research.
Table 4.
Publication venues for the literature sample.
Domain of Publication
Number of Publications
Percentage
Information Systems
12
30%
Computing
9
22.5%
Marketing
9
22.5%
Communication and
Electronic Media
4
10%
Others
6
15%
TOTAL
40
100%
The most suitable research methods to analyze this research topic were experiments
(more than half of the studies) of customer–CA interaction followed by questions related to
the interaction experience, both in real life or simulated settings, see Table
5
.
CA are used for customer service in different industries.
Figure
5
shows that the finan-
cial domain (banking and financial investment) is the most researched field (11 papers) for
the topic of customer experience with customer service AI conversational agents/chatbots.
Electronics
2022
,
11
, 1579
12 of 24
Table 5.
Research approaches.
Research Method
Number of
Publications
Research Method
Number of
Publications
Experiments with
questions
22
Real life
22
Survey
7
Simulations
18
Data analysis
(dialogues)
5
Qualitative
(interviews)
3
Combined
3
TOTAL
40
40
11
8
7
3
2
9
Finance
Hospitality and travel
Retailing
Entertainment
Telecommunication
Multiple industries
Figure 5.
Industry researched.
4.2.
Thematic Analysis—Narrative Description
Table
6
presents details on the seven constructs of the theoretical framework proposed
(Figure
1
)
based
on
the
analysis
of
the
literature
sample.
Table
6
includes
the
related
publications for each construct, the main findings in relationship with each construct and
implications.
The construct influencing factors of CA use has three sub-categories:
CA-
related,
user-related,
and context-related factors that are seen (according to the studied
literature) as drivers for customer experience with this technology for customer service.
Table 6.
Overall customer experience with AI conversational agents/chatbots—summary.
Construct
Publications
Main Results
Implications
Influencing factors
A.
AI CA/chatbot related
influencing factors
-> CA functional
features
(7 studies)
Zarouli et al.
[
44
]; Van den Broeck
et al.
[
45
]; Khadpe et al.
[
46
];
Schuetzler et al.
(2020) [
9
]; Følstad
and Taylor [
5
]; Grundner and
Neuhofer [
47
]; Ringfort-Felner
et al.
[
48
]
Functional features of AI
CA/chatbots, such as
response relevance, tailored
responses, response
understandability, dialogue
outcome (the user received
the needed support), dialogue
efficiency (low time and
effort), competence (error free
interaction), helpfulness,
usefulness, ease of use
represent the key drivers for
positive customer experience.
Companies should emphasize
with priority on functional
features when designing and
using CA/chatbots for
customer service.
Make
customers aware of the ease of
use of CA.
Electronics
2022
,
11
, 1579
13 of 24
Table 6.
Cont.
Construct
Publications
Main Results
Implications
-> CA system features
(8 studies)
Følstad et al.
(2018) [
49
];
Trivedi [
50
]; Luo et al.
[
51
];
Meyer-Waarden et al.
[
52
]; Borsci
et al.
[
53
]; Nguyen et al.
[
54
];
Bührke et al.
[
55
]; Grundner and
Neuhofer [
47
]
System-related features, such
as accessibility of CA/chatbot
functions, reliability (constant
accuracy), service quality has
a positive influence on
customer experience and its
trust.
At the same time,
chatbot identity disclosure has
rather a negative impact on
consumers’ intentions.
System features are to be
considered for improving
quality of the service (chatbot
training).
At the same time,
the dilemma about
transparency related to
CA/chatbot identity needs to
be considered.
-> CA anthropomorphic
features
(16 studies)
Andrews [
56
]; Borsci et al.
[
53
],
De Cicco et al.
[
57
]; Ischen
et al. [
58
]; Meyer-Waarden
et al.
[
52
]; Adam et al.
[
59
]; Crolic
et al.
[
60
]; Bührke et al.
[
55
];
Danckwerts et al.
[
61
]; Ng
et al. [
62
]; Ordemann et al. [
63
]
Chaves et al.
[
64
]; Mehra [
65
];
Toader et al.
[
66
]; Schroeder and
Schroeder [
67
]; Svikhnushina
et al. [
68
]
Studies present contradictory
results in relationship with the
effects of the
anthropomorphic features on
customer experience.
Certain anthropomorphic
features were found to have
no effects on customers’
perceptions and behaviors
(empathy, visual aspect, an
extrovert personality of CA).
Other findings illustrated that
social presence, human-like
design, identity, small talk
have a positive influence on
trust, enjoyment, and
customer satisfaction.
Among
unfavorable effects identified
are that anthropomorphic
features of the AI CA/chatbot
can harm companies, when
consumers are in an angry
state at the time of interaction.
The conclusion is that the
effects of such features need to
be interpreted in correlation
with the context of customer
experience.
The decision on the inclusion
or not of the anthropomorphic
features for CA and on what
type of anthropomorphic
features to be included, needs
to be correlated with the type
of product assisted by the CA,
with the customers’
characteristics and with the
context in which the AC
is used.
B.
User-related
influencing factors
(8 studies)
Andrews [
69
]; Følstad et al.
(2018) [
49
]; De Cicco et al.
[
57
];
Cheng and Jiang [
70
];
Meli
á
n-Gonz
á
lez et al.
[
71
];
Svikhnushina and Pu [
72
];
Tsekouras et al.
[
73
]; Sonntag
et al. [
74
]
Factors related to the
customers that can influence
their experiences with AI
CA/chatbots are of two types:
(a) customer characteristics,
such as age, personality,
expectations, and (b) customer
relationship with technology,
such as personal interest in
technology, previous
experience with the
technology, openness to
innovation, media, and
technology appeal to
customers.
Companies can build profiles
of customers both who are
prone of using the CA
technology for customer
service and who are reluctant
in doing so, by using both
customer characteristics and
customer relationship with
technology.
Electronics
2022
,
11
, 1579
14 of 24
Table 6.
Cont.
Construct
Publications
Main Results
Implications
C.
Context-related
influencing factors
(6 studies)
Følstad et al.
(2018) [
49
];
Trivedi [
50
]; Xu et al.
[
75
]; Cheng
and Jiang [
70
]; Brüggemeier and
Lalone [
76
] Taehyee et al.
[
77
]
Contextual and environmental
factors can also affect the
customer experience with AI
CA. General privacy and
security conditions, especially
in sensitive fields such as
banking can have a negative
influence on the experience.
At the same time, the
company’s image and brands
contribute to trust building
and positive experiences.
Context-related factors are
business and company-related
and have to be identified
individually by each company
using the CA technology for
customer service.
Customers’
perceptions/attitudes/
feelings—positive
(14 studies)
Zarouli et al.
[
44
]; Følstad et al.
(2018) [
49
]; De Cicco et al.
[
57
];
Schuetzler et al.
(2020) [
9
]; Kvale
et al.
[
56
]; Ischen et al.
[
58
];
Hildebrand and Bergner [
78
];
Borsci et al.
[
53
]; Nguyen
et al. [
54
]; Brüggemeier and
Lalone [
76
]; Toader et al.
[
66
];
Svikhnushina and Pu [
72
];
Schroeder and Schroeder [
67
];
Tsekouras et al.
[
73
]
Customer experiences with AI
CA/chatbots can results in
positive perceptions/attitudes
and feelings.
- Trust in CA can be built by
information quality, system
quality, service quality, but
also by the conversational
capacity of CA.
- Enjoyment and fun are
determined by experiential
perceptions and two-way
communication with CA and
by social presence.
- Pleasure and arousal when
using CA are determined by
humanness and social
presence.
- Perceived usefulness is
influenced by accurate and
timely service.
- Benevolence towards the
company appears due to
positive customer experience
with AI CA.
In order to obtain and increase
customer satisfaction and
other positive attitudes and
feelings, companies need to
optimize customer experience
with CA.
Customers’
perceptions/attitudes/
feelings—negative
(9 studies)
Følstad et al.
(2018) [
49
]; Van den
Broeck et al.
[
45
]; Kvale et al.
[
56
];
Cheng and Jiang [
70
];
Meli
á
n-Gonz
á
lez et al.
[
71
];
Chaves et al.
[
64
]; Schuetzler et al.
(2019) [
79
]; Tsekouras et al.
[
73
];
Sonntag et al.
[
74
]
Studies show that interaction
with AI CA can also generate
negative
perceptions/attitudes and
feelings.
- Perceived high risks that can
diminish intention to use AI
CA/chatbots.
- Privacy risks reduce the level
of customer satisfaction.
- Perceived intrusiveness can
have a negative effect on
consumers’ attitudes.
- A low customer satisfaction
is encountered when
CA/chatbots offer generic
responses to a request.
- Inconvenience of using
chatbots (new way of
communication).
Negative perceptions,
attitudes and feelings when
using CA/chatbots, have to be
studied and known by
companies in the first place, in
order to be able to deal with
them.
The privacy issues
represent the most important
aspect to be dealt with for
diminishing negative feelings.
Electronics
2022
,
11
, 1579
15 of 24
Table 6.
Cont.
Construct
Publications
Main Results
Implications
Customers’ responses and
behaviors related to the
CA
(11 studies)
Luo et al.
[
51
]; Xu et al.
[
75
];
Ischen et al.
[
58
]; Hildebrand and
Bergner [
78
]; Nguyen et al.
[
54
];
Brüggemeier and Lalone [
76
];
Stanley et al.
[
80
]; Ng et al.
[
62
];
Ordemann et al.
[
63
];
Svikhnushina and Pu [
72
]; Presti
et al.
[
81
]
The customers’ responses and
behaviors as part of customer
experience with AI
CA/chatbots manifests both,
as intentions and as actions
and behaviors.
Intentions and actions can be
related to the technology itself,
the AI CA. Certain factors
determine the intention to
continue to use AI CA
(tangibles, competence,
reliability of chatbots, trust,
perceived usefulness).
In terms of actions, there are:
the re-use of chatbot
technology, a higher
acceptance of the AI CA
recommendations and
advices, and the
recommendation of the
chatbot use to other
customers.
Companies need to identify
the specific factors that have a
positive influence on the
customers’ intention to re-use
the CA and their higher
compliance to the CA
recommendations and focus
on those.
Customers’ responses and
behaviors—related to the
company
(7 studies)
Trivedi [
50
]; Van den Broeck
et al. [
45
]; Luo et al.
[
51
]; Khadpe
et al. [
46
]; Cheng and Jiang [
70
];
Hildebrand and Bergner [
78
];
Danckwerts et al. [
61
]
Intentions and actions of
customers based on
customers’ experience with AI
CA can manifest towards the
company, as well.
Reactions
can be both positive and
negative.
- Benevolence towards the
company is determined by
high conversational skills of
CA.
-Patronage intentions (buy
and recommend the
company’s product) are
influenced by the trust in
CA/chatbots, by social
presence and competence of
CA/chatbots, by perceived
usefulness, helpfulness and
relevance of the CA/chatbots’
answers.
-In addition, loyalty to brands
is influenced by customer
satisfaction and love for
brands is influenced by the
CA/chatbot success
(information, system and
service quality).
-Negative reactions to AI
CA/chatbots were
encountered when consumers
know that the conversational
partner is not human, they
purchase less.
Companies need to be aware
of both:
(a) the effect of the
use of AI CA technology for
customer service on the
company’s image and brands
and, (b) vice versa, the effect
of the company’s image and
brand on the perception of the
CA used for customer
services.
Electronics
2022
,
11
, 1579
16 of 24
Based on the literature sample, three new sub-categories were identified for CA related
factors, namely the functional features of CA, the system features, and the anthropomorphic
features of CA. The constructs customers’ perceptions/attitudes and feelings (with the two
sub-categories positive and negative) and the customers’ responses and behaviors (with
the two sub-categories:
related to the CA and related to the company) represent results and
components of the overall customer experience.
The narrative description intents to answer the research questions of the study using
the findings presented in the analyzed publications.
The analysis is based on the theoretical
framework proposed for the analysis (Figure
1
).
4.2.1.
Influencing Factors of AI CA/Chatbot Use
RQ1.:
What are the factors that influence the customer experiences with AI CA/chatbots for
customer service?
Some studies considered and analyzed just one influencing factor (CA/chatbot social
presence [
57
], CA/chatbot personality [
69
], problem resolution capacity [
56
], while others
looked at a combination of influencing factors [
5
,
45
].
Influencing factors for the overall
customer experience with AI CA/chatbots were grouped in the literature in three major
categories [
8
] (chatbot-related, user-related, and context-related) and our analysis used this
typology to discuss the factors that influence customer experience.
The factors that are related to the technology itself, in our case the AI CA/chatbot, are
numerous and they can also be further grouped.
The typology we propose for grouping
the CA/chatbot related factors include (a) factors related to functional features; (b) factors
related to the system’s features, and (c) anthropomorphic factors.
Among the functional features, Borsci et al., for example, found as main factors that
influence customer experience with AI CA/chatbots:
perceived quality of the chatbot func-
tions, perceived quality of the conversation and information provided, perceived privacy
and
security
and
time
response.
In
addition,
in
their
study,
the
perceived
accessibility
to
chatbot
functions
was
another
influencing
factor
that
is
part
of
the
system’s
feature
category [
53
].
Similarly,
based
on
a
survey
conducted
in
France
related
to
the
use
of
a
French
CA/chatbot from the airline industry (FlyBot), Meyer-Waarden et al.
showed that when
chatbots for customer service provide consumers with relevant, reliable, and functional
content, it positively impacts future intentions to re-use the technology.
At the same time,
non-instrumental factors such as empathy were found not to be relevant for automated
customer service involving routine interaction [
52
].
Anthropomorphic features have been extensively studied in order to identify their
influences on customers’ reactions.
In an experiment conducted in Germany with bank
customers, Adam et al.
illustrated that anthropomorphic cues, such as identity, small-talk
and
empathy,
positively
influence
the
customer’s
behaviour
by
encouraging
customer
compliance with the AI CA/chatbot request [
59
].
In lab experiments with US students,
Schuetzler
et
al.
found
that
conversational
skills
of
CA
(manifested
through
tailored
responses
and
through
variety
of
responses)
increase
the
social
presence
perceived
by
customers
and,
therefore,
the
perceived
anthropomorphism
of
CA
[
9
].
In
other
words,
customers
perceive
a
CA/chatbot
that
has
higher
conversational
skills
as
being
more
human-like and with a higher level of engagement than one that has lower conversational
skills (offers more generic and non-varied responses.
In a research study conducted in Italy
with millennials, De Cicco et al.
identified as an influencing factor of customer experience,
the
social
presence
of
chatbots,
which
was
defined
by
visual
cues
(avatar/non-avatar)
and
interaction
style
(social-oriented
or
task-oriented)
that
had
a
positive
influence
on
customers’ feelings [
57
].
However, the anthropomorphic traits, in certain circumstances have been found to
have
rather
negative
influences
on
the
customer
experience.
Based
on
an
extensive
set
of real life data of customer interaction (34,639 entries) and four experiments (more than
1000 participants in total), Crolic et al.
found that when customers enter interaction with
Electronics
2022
,
11
, 1579
17 of 24
an
anthropomorphic
chatbot
in
an
angry
state
(in
a
customer
complain
situation,
for
example)
(a
context-related
factor),
there
is
a
higher
probability
that
the
customer
will
be less satisfied with the interaction, will evaluate at a lower level the company, and its
purchase intentions will diminish.
These negative responses relate to the violation of the
high pre-interaction expectations in terms of chatbot efficacy that consumers have from
chatbots with anthropomorphic features [
60
].
There were studies that considered a larger number of influencing factors, combin-
ing the different categories of factors.
Meli
á
n-Gonz
á
lez et al.
conducted a survey with
476 young Spanish tourists who have interacted with CA/chatbots for their travel pur-
poses and revealed four factors that positively influenced their intention to use chatbots
for tourism:
(a) the performance expectancy when using chatbots (CA/chatbot related—
functional); (b) consumers’ habit of using chatbot technology (user-related); (c) consumers’
hedonism motivations (user-related), and d) the social presence depicted in chatbot inter-
action (CA/chatbot-related—anthropomorphism).
At the same time, the inconveniences
associated with chatbot use (such as the need of adapting to a new communication style)
have a negative influence on their intention to use chatbots in the future [
71
].
4.2.2.
Resulting Dimensions of Customer Experience
RQ2.: What are the resulting dimensions of the overall customer experience with AI CA/chatbots
for customer service?
This research question aimed to identify the resulting dimension of the overall cus-
tomer experiences in relationship with the use of AI customer service CA/chatbots.
Accord-
ing to the theoretical framework, there are two types of resulting dimensions:
(a) feelings,
attitudes, and perceptions (that can be positive or negative) and (b) responses and behaviors
(that can include intentions and actions and can be directed towards the CA/chatbots or
towards the company).
A.
Perceptions, attitudes, and feelings
The perceptions, attitudes, and feelings of customers when using AI CA/chatbots for
customer service can be both positive and negative, depending on the various previously
presented factors and on the different circumstances in which the experience takes place.
Love
for
a
brand
was
a
feeling
that
was
studied
by
researchers
in
the
context
of
CA/chatbot use [
50
].
Generation Y Indian consumers of banking companies who inter-
acted with customer service chatbots offered by banks, reported rather positive customer
experiences with CA/chatbots (influenced by the service quality, information quality, and
system quality of chatbots) that further increased their brand love for the bank brands that
use CA/chatbots as opposed to the ones that do not use the technology.
In
addition,
De
Cicco
et
al.
found
that
for
millennials
in
Italy,
a
social-oriented
conversational style induces the feeling of social presence for AI CA/chatbots, higher levels
of
trust
and
perceived
enjoyment
when
using
them
and,
in
addition,
drives
a
positive
customer attitude towards AI CA/chatbots [
57
].
However,
experiences
with
AI
CA/chatbots
determined
also
negative
feelings
for
some customers.
Kvale et al.
found that when CA/chatbots offer too generic information
as opposed to information that directly follows the customers’ request, the result is low
customer satisfaction [
56
]. In another study, Chen and Jiang illustrated that a high perceived
privacy risk diminishes the customer satisfaction with CA [
70
].
Some customers reported
inconveniences and difficulties when interacting with CA because they consider that the
relationship cannot be based on natural language [
71
].
Another feeling affecting negatively
the patronage intentions of customers is the perceived intrusiveness of ads displayed by
CA [
45
].
B.
Responses and behaviors
In addition, responses and behaviors of consumers as part of customer experience with
CA/chatbots was a topic extensively approached in research studies.
Such responses and
behaviors can be categorized according to two criteria:
(a) the type of reaction (intention
Electronics
2022
,
11
, 1579
18 of 24
and
actual
behavior)
and
(b)
the
recipient
of
the
responding
behavior
(the
technology
itself—CA/chatbot and the company offering the service).
Accordingly, there are inten-
tions
towards
the
CA/chatbot
(the
intention
to
re-use
the
CA
for
customer
service
or
not) [
52
,
70
,
71
,
75
] and intentions towards the company (the intention to buy or re-buy from
the company) [
81
].
At the same time,
there are behaviors towards the CA/chatbot (the
use and re-use of the technology, complying with the CA recommendations) [
54
,
58
,
59
,
78
]
and behaviors towards the company (patronage intentions, buying the company’s prod-
ucts, recommending the company’s products) [
44
,
51
,
66
].
For example, when chatbots for
customer service provide consumers with relevant, reliable, and functional content, they
support the users’ intention to re-use the customer service CA/chatbot [
52
].
Studies tried to make the connection between influencing factors, attitudes, and feel-
ings
of
customers
when
using
customer
service
CA
and
resulting
behaviors
related
to
CA use.
In a research paper that conducted 4 experimental studies with over 300 active
investors in Switzerland, Hildebrand and Bergner empirically tested the influence of con-
versational robo-advisors versus influence of the non-conversational robo-advisors on the
perceptions
and
the
behavior
of
investors.
They
demonstrated
that
the
conversational
capacity of a robo-advisor (with or without social cues) offers a more engaging user ex-
perience
during
the
investors’
acquisition
stage,
with
a
positive
influence
first
on
their
perceptions (higher affective trust) and based on these, subsequently, on their behaviors (ac-
cepting recommendations) for investment from conversational robo-advisors, as compared
to the non-conversational ones [
78
].
Another study [
66
] was based on a simulated experiment in which 240 US participants
interacted with a prototype commercial site for apparel products that included CA/chatbots.
The research revealed that both users’ perceptions on social presence and competence of
the chatbot play a critical role in developing strong trusting beliefs and that trust further
determines a positive effect on purchase intentions.
Patronage intentions as a result of customers’ experience with CA/chatbots was also
approached in a number of studies [
44
,
45
,
66
] considering both the intention to buy the
company’s product and to recommend it to others.
One particular category of studies were the comparative studies, in which CA/chatbots
for customer service were compared to human interaction for customer service or to other
IT technologies for the customer service [
51
,
58
,
75
,
77
].
For example, based on a real-life field
experiment conducted with Asian customers, Luo et al.
illustrated that AI CA/chatbots
are equally effective as proficient workers and four times more effective than workers with
less experience, in generating sales for a FinTech company (for renewing loans) (product
purchase behavior) [
51
].
Results
and
learnings
of
such
studies
can
be
applied
for
the
improvement
of
the
activity of CA, first of all, in customer service of businesses [
15
], but also for other types of
activities or domains [
82
,
83
].
The
cross-study
synthesis
applied
the
theoretical
framework
proposed,
in
which
influencing
factors
and
overall
customer
experience
dimensions
were
put
together
(as
presented in Table
6
).
As a conclusion, all elements of the theoretical framework are part of
a logical sequence in the customer experience flow and are interconnected: first, influencing
factors enter and contribute to the interaction customer–AI CA/chatbot; second, as a result
of the experience, customers have certain perceptions, attitudes, and feelings regarding
CA/chatbot interaction and, third, further on the feelings and perceptions determine certain
behaviors (related to the chatbot itself and related to the company and its products).
5.
Conclusions
This paper aimed to identify the main influencing factors for customer experience with
customer service AI CA/chatbots, as well as to analyze the customer perceptions, attitudes
and feelings related to AI CA/chatbot use, on the one hand and the customers’ responses
and behaviors on the other hand.
The systematic literature review method was used for
Electronics
2022
,
11
, 1579
19 of 24
this purpose and, based on Okoli’s [
37
] SLR methodology, 40 empirical publications were
included in the analysis.
The
main
ideas
that
emerge
related
to
overall
customer
experience
with
customer
service AI CA/chatbots are:
1.
There is a large variety of influencing factors of AI CA/chatbot use, as well as per-
ceptions, attitudes and feelings and also responses and behaviors that are related to
customer experience as presented in Table
6
.
The influencing factors can be grouped
in three major categories:
factors related to the CA/chatbot itself, factors related to
the user, and factors related to situational context.
In addition, the AI CA/chatbot-
related factors can be further categorized in functional features, system features, and
anthropomorphic features.
The factors’ effects on customer experience can be both
positive or negative.
2.
The most relevant influencing factors for obtaining customer satisfaction with cus-
tomer services as part of the customer experience are the functional and utilitarian
features of AI CA/chatbots that impact their performance.
When AI CA/chatbots
function
and
perform
well
(in
terms
of
capability
to
understand
the
request,
rele-
vance of the responses offered, solving the customer’s request, bring time and effort
economy for customer), they are perceived as being competent and reliable.
In these
circumstances, they always have a positive influence on customer experience with AI
CA/chatbots.
At the same time, solving the consumer’s task and offering relevant
information diminishes other potentially negative perceptions on AI CA/chatbots,
such as intrusiveness or lack of privacy.
3.
One important influencing factor that was highly analyzed by researchers relates to
the anthropomorphism of AI CA/chatbots and its effects on customer experience.
The
anthropomorphic features of AI CA/chatbots can have both positive and negative
effects
on
customer
experience.
According
to
a
number
of
studies,
anthropomor-
phic characteristics with positive effects on customer experience are:
female gender
CA/chatbots are found to be positively perceived; social presence and social inter-
action
positively
influence
young
consumers,
as
communication
and
interactivity
creates enjoyment.
However, other studies concluded that anthropomorphic features
can harm companies.
When consumers enter interaction with the AI CA/chatbot in
an anger state (in a customer complain situation), the existence of anthropomorphic
cues of the CA/chatbot induces higher efficacy expectations of the consumer from the
CA. In addition, if the anthropomorphic CA cannot fulfil appropriately the tasks, the
customers’ satisfaction diminishes and their purchase intention also decreases.
The
diverse and also contradictory results illustrate that customer experience is highly
dependent on circumstances, as well.
4.
It can be stated that the contextual influencing factors also contribute to the customer
experience.
Among those, another important moderating factor of the relationship
CA/chatbot–customer that appears frequently in research studies, refers to privacy
issues.
Results illustrate that privacy assurance can have positive effects on customers’
experience, up to higher degrees of product purchase.
At the same time, perceived
high privacy risks have negative effects on customers’ attitudes, especially for privacy
sensitive domains, such as financing (banking, investment).
5.
In many industries, customer service chatbots perform very well and are very well
perceived by consumers (in terms of utility, helpfulness, time and effort) when fulfill-
ing low-complexity tasks.
At the same time, task-oriented chatbots (as opposed to
social-oriented chatbots) have been found in more studies to have a higher level of
suitability in case of customer services.
6.
The
use
of
diverse
customer
service
AI
CA/chatbots
can
determine
both
positive
feelings (such as satisfaction, trust, enjoyment, pleasure) but also negative feelings
(such as distrust, intrusion, inconvenience) for customers, depending on the effects of
the three major types on influencing factors (CA/chatbot-related, customer-related
factors and context/environment-related factors) on customers’ overall experience.
Electronics
2022
,
11
, 1579
20 of 24
7.
The effects of the use of customer service AI CA/chatbots on customers’ responses
and behavior manifest in two directions and for two types of responses:
first, towards
the
chatbot
itself
(intention
and
usage
continuation
or
not)
and,
second,
towards
the company and the brand (intention and product purchase and recommendation
or not).
Practical implications
.
The results of the present study have important practical im-
plications for AI CA/chatbots designers and generally for companies willing to integrate
AI/CA chatbot technology in their customer service activity.
CA/chatbot designers have to
consider those features that are well perceived by customers.
In terms of anthropomorphic
features, female identity and voice and social presence are important positive influencers,
when targeting young customers, for example.
At the same time, functional features that
allow for solving the task are of paramount importance for all types of customers using
AI CA for customer service.
However, designers can avoid those features that are either
negatively perceived and/or have no contribution at all, such as empathy when the task is
simple information provision.
Companies integrating AI CA/chatbot technology can use the technology for low-
complexity customer services tasks, while for high-complexity tasks, they can offer a com-
bination of computerized followed by human-assisted services or solely human-assisted
services.
In addition, companies should focus on using customer service AI CA/chatbots
for solving those tasks for which a very concrete and relevant response can be provided via
this technology.
Theoretical implications and future research
.
Previous studies and reviews approached AI
CA/chatbots from different perspectives, some technical, some historical, some looking
at AI CA/chatbot use and user experience by considering just one or a limited number
of
influencing
factors.
The
present
paper
contributes
with
a
comprehensive
synthesis
on the numerous influencing factors for customer experience with customer service AI
CA/chatbots
and
also
the
resulting
customers’
feelings
and
behaviors
as
presented
in
empirical research and proposed a theoretical framework that integrates them.
The
contribution
of
the
paper
consists
in
using
an
overall
approach
to
customer
experience with customer service AI CA/chatbots that follows the logic of identifying the
influencing factors for the use of the CA; then, we see what are the resulting dimensions of
customer experience with CA in terms of customers’ reactions. The multitude of influencing
factors were first grouped by looking at the existing typologies in the literature (CA-related,
user-related, and environment-related), but the study further proposes a more detailed way
for categorizing the CA/chatbot-related factors (in functional features, system features,
and
anthropomorphic
features).
The
two
resulting
dimensions
of
customer
experience
(feelings and behaviors) were also categorized in this paper, illustrating the diversity of
outcomes that can result during and after customers’ experience with AI CA/chatbots.
The
customers’ perceptions/attitudes and feelings when using AI CA/chatbots can be both
favorable (satisfaction, pleasure, enjoyment) and unfavorable (distrust, inconvenience).
The
paper also proposes the categorization of customers’ reactions when using AI CA/chatbots
based on two criteria:
the recipient of the reaction (the CA itself and the company) and the
type of reaction (intention and action).
The proposed framework is a good tool to be used
to analyze the customer experience with this technology (CA) in the context of customer
service for different industries and its use can be extended to other types of IT technologies,
as well.
Future research can extend to customer service CA/chatbots used in other domains,
such as public services, for example, education or health or local authorities’ services.
At
the same time, the analysis of customer experience with other types of AI CA/chatbots,
such as social companions and personal voice assistance is another interesting direction for
further research.
Limitations
. The limitations of the study relate to the small research team and associated
time restrictions.
Another limitation relates to the fact that the paper considers only the
Electronics
2022
,
11
, 1579
21 of 24
business domain to look at customer service AI CA/chatbots, while the technology applies
to other domains as well.
Author Contributions:
Conceptualization, M.T.T. methodology, M.T.T.; assisting in validation, L.N.;
investigation, M.T.T.; resources, M.T.T.; writing—original draft preparation, M.T.T.; writing—review,
M.T.T.; editing and revision of review L.N.; supervision of final form of the paper, L.N. All authors
have read and agreed to the published version of the manuscript.
Funding:
This research received no external funding.
Conflicts of Interest:
The authors declare no conflict of interest.
Appendix A
Table A1.
Extraction form for users’ experience (UX) with customer service AI CA and chatbots.
Data to Be Extracted and Evaluated
Reviewer Notes
Title of the publication
Journal
Journal domain
Author(s)
Authors’ origin (country and institution)
Year of publication
Setting (town/country/continent)
Industry
Time of data collection
M: Objective of the study
M: Research question(s)
M: Study design (quantitative, qualitative, combined)
M: Research methods (survey, experiments, etc.)
M: Sources of data
M: Sample characteristics (participants) and sample size
M: Research instruments
M: Data analysis methods
V: Influencing factors for customer experience (UX)
V: Feelings/attitudes/perceptions of customers
V: Responses and behaviors of customers
V: Benefits and challenges of using AI CA and chatbots
R: Main findings
R: Implications
R: Conclusions
M—methodology; V—variables studied; R—results.
References
1.
Dale, R. The return of the chatbots.
Nat.
Lang.
Eng.
2016
,
22
, 811–817.
[
CrossRef
]
2.
Skjuve, M.; Haugstveit, I.M.; Følstad, A.; Brandtzaeg, P.B. Help!
Is my chatbot falling into the uncanny valley?
An empirical
study of user experience in human-chatbot interaction.
Hum.
Technol.
2019
,
15
, 30–54.
[
CrossRef
]
3.
Adamopoulou, E.; Moussiades, L. Chatbots:
History, technology, and applications.
Mach.
Learn.
Appl.
2020
,
2
, 100006.
[
CrossRef
]
4.
Følstad, A.; Brandtzaeg, P.B. Users’ experience with chatbots:
Findings from a questionnaire study.
Qual.
User Exp.
2020
,
5
, 3.
[
CrossRef
]
5.
Følstad, A.; Taylor, C. Investigating the user experience of customer service chatbot interaction:
A framework for qualitative
analysis of chatbot dialogues.
Qual.
User Exp.
2021
,
6
, 6.
[
CrossRef
]
6.
Ultimate.
Available online:
ultimate.ai
(accessed on 5 March 2022).
7.
Wilkinson,
D.;
Alkan,
O.;
Liao,
Q.V.;
Mattetti,
M.;
Vejsbjerg,
I.;
Knijnenburg,
B.P.;
Daly,
E.
Why
or
why
not?
The
effect
of
justification styles on chatbot recommendations.
ACM Trans.
Inf.
Syst.
2021
,
39
, 42.
[
CrossRef
]
8.
Nordheim, C.B.; Følstad, A.; Bjorkli, C.A. An initial model of trust in chatbots for customer service—findings from a questionnaire.
Interact.
Comput.
2019
,
31
, 317–335.
[
CrossRef
]
9.
Schuetzler, R.M.; Grimes, G.M.; Giboney, J.S. The impact of chatbot conversational skill on engagement and perceived humanness.
J. Manag.
Inf.
Syst.
2020
,
37
, 875–900.
[
CrossRef
]
Electronics
2022
,
11
, 1579
22 of 24
10.
Amico, M.D.; Zikmund, W.G.
The Power of Marketing. Creating and Keeping Customers in an E-commerce World
, 7th ed.; South-Western
College Publishing:
Cincinnati, OH, USA, 2001.
11.
Brandtzaeg, P.B.; Følstad, A. Chatbots:
Changing user needs and motivations.
Interactions
2018
,
25
, 38–43.
[
CrossRef
]
12.
Følstad, A.; Araujo, T.; Law, E.L.C.; Brandtzaeg, P.B.; Papadopoulos, S.; Reis, L.; Baez, M.; Laban, G.; McAllister, P.; Ischen, C.;
et al.
Future directions for chatbot research:
An interdisciplinary research agenda.
Computing
2021
,
103
, 2915–2942.
[
CrossRef
]
13.
Suhaili, S.M.; Salim, N.; Jambli, M.N. Service chatbots:
A systematic review.
Expert Syst.
Appl.
2021
,
184
, 115461.
[
CrossRef
]
14.
Rheu, M.; Shin, J.Y.; Peng, W.; Huh-Yoo, J. Systematic review:
Trust-building factors and implications for conversational agent
design.
Int.
J. Hum.
Comput.
Interact.
2021
,
37
, 81–96.
[
CrossRef
]
15.
Jenneboer, L.; Herrando, C.; Constantinides, E. The Impact of Chatbots on Customer Loyalty:
A Systematic Literature Review.
J.
Theor.
Appl.
Electron.
Commer.
Res.
2022
,
17
, 212–229.
[
CrossRef
]
16.
Alam, R.; Islam, M.A.; Khan, A.R. Usage of chatbot as a new digital communication tool for customer support:
A case study on
Banglalink.
Indep.
Bus.
Rev.
2019
,
12
, 31–37.
Available online:
http://www.sbe.iub.edu.bd/wp-content/uploads/2020/09/a4.pdf
(accessed on 10 December 2021).
17.
Ho, A.; Hancock, J.; Miner, A.S. Psychological, Relational, and Emotional Effects of Self-Disclosure After Conversations with a
Chatbot.
J. Commun.
2018
,
68
, 712–733.
[
CrossRef
] [
PubMed
]
18.
Riikkinen, M.; Saarijarvi, H.; Sarlin, P.; Lahteenmaki, I. Using artificial intelligence to create value in insurance.
Int.
J. Bank Mark.
2018
,
36
, 1145–1168.
[
CrossRef
]
19.
Huang, D.H.; Chueh, H.E. Chatbot usage intention analysis:
Veterinary consultation.
J. Innov.
Knowl.
2021
,
6
, 135–144.
[
CrossRef
]
20.
Gao, C.; Lei, W.; He, X.; Rijke, M.; Chua, T.S. Advances and challenges in conversational recommender systems:
A survey.
AI
Open
2021
,
2
, 100–126.
[
CrossRef
]
21.
Zierau, N.; Wambsganss, T.; Janson, A.; Schöbel, S.; Leimeister, J.M. The anatomy of user experience with conversational agents:
A taxonomy and propositions of service clues.
In Proceedings of the 2020 41st International Conference on Information Systems
(ICIS), Hyderabad, India, 13–16 December 2020.
Available online:
https://www.alexandria.unisg.ch/261080/1/JML_785.pdf
(accessed on 5 March 2022).
22.
Verhagen, T.; Van Nes, J.; Feldberg, J.; Van Dolen, W. Virtual Customer Service Agents:
Using Social Presence and Personalization
to Shape Online Service Encounters.
J. Comput.
Mediat.
Commun.
2014
,
19
, 529–545.
[
CrossRef
]
23.
Fogli, L.
Customer Service Delivery:
Research and Best Practices
, 3rd ed.; Jossey-Bass:
San Francisco, CA, USA, 2006.
24.
Nißen,
M.;
Selimi,
D.;
Janssen,
A.;
Rodr
í
guezCardona,
D.;
Breitner,
M.H.;
Kowatsch,
T.;
von Wangenheim,
F. See you again,
chatbot?
A design taxonomy to characterize user-chatbot relationships with different time horizons.
Comput.
Hum.
Behav.
2022
,
127
, 107043.
[
CrossRef
]
25.
Grewal, D.; Herhausen, D.; Ludwig, S.; Ordenes, F.V. The future of digital communication research:
Considering dynamics and
multimodality.
J. Retail.
2021,
in press
.
[
CrossRef
]
26.
Reinartz, W.; Wiegand, N.; Imschloss, M. The impact of digital transformation on the retailing value chain.
Int.
J. Res.
Mark.
2019
,
36
, 350–366.
[
CrossRef
]
27.
ISO 9241-210-2019 E
; Ergonomics of Human System Interaction—Part 210:
Human-Centered Design for Interactive Systems.
International Organization for Standardization (ISO): Geneva, Switzerland, 2019.
Available online:
https://www.iso.org/obp/
ui/#iso:std:iso:9241:-210:ed-2:v1:en
(accessed on 10 December 2021).
28.
Venkatesh, V.; Thong, J.Y.L.; Xu, X. Consumer Acceptance and Use of Information technology:
Extending the Unified Theory of
Acceptance and Use of Technology.
MIS Q.
2012
,
36
, 157–178.
[
CrossRef
]
29.
Hassenzahl, M.; Schobel, M.; Trautmann, T. How Motivational Orientation Influences the Evaluation and Choice of Hedonic and
Pragmatic Interactive Products:
The Role of Regulatory Focus.
Interact.
Comput.
2008
,
20
, 473–479.
[
CrossRef
]
30.
Blut, M.; Wang, C.; Wünderlich, N.V.; Brock, C. Understanding anthropomorphism in service provision:
A meta-analysis of
physical robots, chatbots, and other AI.
J. Acad.
Mark.
Sci.
2021
,
49
, 632–658.
[
CrossRef
]
31.
Berry, L.L.; Wall, E.A.; Carbone, L.P. Service Clues and Customer Assessment of the Service Experience:
Lessons from Marketing.
Acad.
Manag.
Perspect.
2006
,
20
, 43–57.
[
CrossRef
]
32.
Bhattacherjee,
A.
Understanding
Information
Systems
Continuance:
An
Expectation-Confirmation
Model.
MIS
Q.
2001
,
25
,
351–370.
[
CrossRef
]
33.
DeLone, W.H.; McLen, E.R. The DeLone and McLean Model of Information Systems Success:
A Ten-Year Update.
J. Manag.
Inf.
Syst.
2003
,
19
, 9–30.
[
CrossRef
]
34.
Kitchenham,
B.;
Charters,
S.
Guideline
for
Performing
Systematic
Literature
Reviews
in
Software
Engineering
Version
2.3.
EBSE
Technical
Report
EBSE-2007-01.
Available
online:
https://www.elsevier.com/__data/promis_misc/525444
systematicreviewsguide.pdf.
(accessed on 15 December 2021).
35.
Rowe, F. What literature review is not:
Diversity, boundaries and recommendations.
Eur.
J. Inf.
Syst.
2014
,
23
, 241–255.
[
CrossRef
]
36.
Templier, M.; Pare, G. Transparency in literature reviews:
An assessment of reporting practices across review types and genres in
top IS journals.
Eur.
J. Inf.
Syst.
2017
,
27
, 503–550.
[
CrossRef
]
37.
Okoli, C. A guide to conducting a standalone systematic literature review.
Commun.
Assoc.
Inf.
Syst.
2015
,
37
, 879–910.
[
CrossRef
]
38.
Webster,
J.;
Watson,
R.
Analyzing
the
past
to
prepare
for
the
future:
Writing
a
literature
review.
MIS
Q.
2002
,
26
,
xiii–xxiii.
Available online:
http://www.jstor.org/stable/4132319
(accessed on 17 April 2022).
Electronics
2022
,
11
, 1579
23 of 24
39.
Collins, C.; Dennehy, D.; Conboy, K.; Mikalef, P. Artificial intelligence in information systems research:
A systematic literature
review and research agenda.
Int.
J. Inf.
Manag.
2021
,
60
, 102383.
[
CrossRef
]
40.
Levy, Y.; Ellis, T.J. A systems approach to conduct an effective literature review in support of information systems research.
Int.
J.
Emerg.
Transdiscipl.
2006
,
9
, 181–212.
[
CrossRef
]
41.
Eismann, K.; Posegga, O.; Fischbach, K. Decision Making in Emergency Management:
The Role of Social Media.
2018 Research
Papers, 152.
Available online:
https://aisel.aisnet.org/ecis2018_rp/152
(accessed on 4 December 2021).
42.
Page,
M.J.;
McKenzie,
J.E.;
Bossuyt,
P.M.;
Boutron,
I.;
Hoffmann,
T.C.;
Mulrow,
C.D.;
Shamseer,
L.;
Tetzlaff,
J.M.;
Akl,
E.A.;
Brennan, S.E.; et al.
The PRISMA 2020 statement:
An updated guideline for reporting systematic reviews.
BMJ
2021
,
372
, n71.
[
CrossRef
]
43.
Petticrew, M.; Roberts, H.
Systematic Reviews in the Social Sciences:
A Practical Guide
; Blackwell Publishing:
Oxford, UK, 2006.
44.
Zarouali, B.; Broeck, E.V.D.; Walrave, M.; Poels, K. Predicting consumer responses to a chatbot on Facebook.
Cyberpsychol.
Behav.
Soc.
Netw.
2018
,
21
, 491–497.
[
CrossRef
]
45.
Van den Broeck, E.; Zarouali, B.; Poels, K. Chatbot advertising effectiveness:
When does the message get through?
Comput.
Hum.
Behav.
2019
,
98
, 150–157.
[
CrossRef
]
46.
Khadpe,
P.;
Krishna,
R.;
Fei-Fei,
L.;
Hancock,
J.T.;
Bernstein,
M.S.
Conceptual
Metaphors
Impact
Perceptions
of
Human-AI
Collaboration.
In
Proceedings
of
the
ACM
on
Human-Computer
Interaction,
New
York,
NY,
USA,
15
October
2020;
p.
163.
[
CrossRef
]
47.
Grundner,
L.;
Neuhofer,
B.
The
bright
and
dark
sides
of
artificial
intelligence:
A
futures
perspective
on
tourist
destination
experiences.
J. Destin.
Mark.
Manag.
2020
,
19
, 100511.
[
CrossRef
]
48.
Ringfort-Felner, R.; Laschke, M.; Sadeghian, S.; Hassenzahl, M.; Kiro, I. A Design Fiction to Explore Social Conversation with Voice
Assistants.
In Proceedings of the ACM on Human-Computer Interaction, Online, 14 January 2022; Volume 6, p.
33.
[
CrossRef
]
49.
Følstad, A.; Nordheim, C.B.; Bjørkli, C.A. What Makes Users Trust a Chatbot for Customer Service?
An Exploratory Interview
Study.
In Proceedings of the International Conference on Internet Science, St.
Petersburg, Russia, 24–26 October 2018; Lecture
Notes in Computer Science; Bodrunova, S.S., Ed.; Springer:
Berlin/Heidelberg, Germany, 2018; Volume 11193.
[
CrossRef
]
50.
Trivedi, J. Examining the Customer Experience of Using Banking Chatbots and Its Impact on Brand Love:
The Moderating Role
of Perceived Risk.
J. Internet Commer.
2019
,
18
, 91–111.
[
CrossRef
]
51.
Luo, X.; Tong, S.; Fang, Z.; Qu, Z. Frontiers:
Machines vs.
Humans:
The Impact of Artificial Intelligence Chatbot Disclosure on
Customer Purchases.
Mark.
Sci.
INFORMS
2019
,
38
, 937–947.
[
CrossRef
]
52.
Meyer-Waarden,
L.;
Pavone,
G.;
Poocharoentou,
T.;
Prayatsup,
P.;
Ratinaud,
M.;
Tison,
A.;
Torne,
S.
How
Service
Quality
Influences Customer Acceptance and Usage of Chatbots?
J. Serv.
Manag.
Res.
2020
,
4
, 35–51.
[
CrossRef
]
53.
Borsci, S.; Malizia, A.; Schmettow, M.; van der Velde, F.; Tariverdiyeva, G.; Balaji, D.; Chamberlain, A. The Chatbot Usability Scale:
The Design and Pilot of a Usability Scale for Interaction with AI-Based Conversational Agents.
Pers.
Ubiquitous Comput.
2022
,
26
,
95–119.
[
CrossRef
]
54.
Nguyen, D.M.; Chiu, Y.T.H.; Le, H.D. Determinants of Continuance Intention towards Banks’ Chatbot Services in Vietnam:
A
Necessity for Sustainable Development.
Sustainability
2021
,
13
, 7625.
[
CrossRef
]
55.
Bührke, J.B.; Brendel, A.B.; Lichtenberg, S.; Greve, M.; Mirbabaie, M. Is Making Mistakes Human?
On the Perception of Typing
Errors in Chatbot Communication.
In Proceedings of the 54th Hawaii International Conference on System Sciences, Maui, HI,
USA, 5–8 January 2021.
Available online:
https://scholarspace.manoa.hawaii.edu/bitstream/10125/71158/0438.pdf
(accessed
on 10 December 2021).
56.
Kvale, K.; Freddi, E.; Hodnebrog, S.; Sell, O.A.; Følstad, A. Understanding the User Experience of Customer Service Chatbots:
What Can We Learn from Customer Satisfaction Surveys?
In Proceedings of the International Workshop on Chatbot Research
and Design, Virtual, 23–24 November 2020; Lecture Notes in Computer Science.
Følstad, A., Araujo, T., Papadopoulos, S., Law,
E.L.-C., Luger, E., Goodwin, M., Brandtzaeg, P.B., Eds.; Springer:
Cham, Switzerland, 2021; Volume 12604.
[
CrossRef
]
57.
De Cicco, R.; Silva, S.C.; Alparone, F.R. Millennials’ attitude toward chatbots:
An experimental study in a social relationship
perspective.
Int.
J. Retail Distrib.
Manag.
2020
,
48
, 1213–1233.
[
CrossRef
]
58.
Ischen, C.; Araujo, T.; van Noort, G.; Voorveld, H.; Smit, E. “I Am Here to Assist You Today”:
The Role of Entity, Interactivity and
Experiential Perceptions in Chatbot Persuasion.
J. Broadcast.
Electron.
Media
2020
,
64
, 615–639.
[
CrossRef
]
59.
Adam, M.; Wessel, M.; Benlian, A. AI-based chatbots in customer service and their effects on user compliance.
Electron.
Mark.
2021
,
31
, 427–445.
[
CrossRef
]
60.
Crolic, C.; Thomaz, F.; Hadi, R.; Stephen, A. Blame the bot:
Anthropomorphism and anger in customer-chatbot interactions.
J.
Mark.
2021
,
86
, 132–148.
[
CrossRef
]
61.
Danckwerts, S.;
Meissner, L.;
Krampe, C. Examining User Experience of Conversational Agets in Hedonic Digital services—
Antecedents and the Role of Psychological Ownership.
J. Serv.
Res.
2019
,
3
, 111–125.
[
CrossRef
]
62.
Ng, M.; Coopamootoo, K.P.L.; Toreini, E.; Aitken, M.; Elliot, K.; Moorsel, A. Simulating the Effects of Social Presence on Trust,
Privacy Concerns & Usage Intentions in Automated Bots for Finance.
In Proceedings of the 2020 IEEE European Symposium on
Security and Privacy Workshops (EuroS&PW), 190-199, Genoa, Italy, 7–11 September 2020.
[
CrossRef
]
63.
Ordermann, S.; Skjuve, M.; Følstad, A.; Bjorkli, C.A. Understanding how chatbots work:
An exploratory study of mental models
in customer service chatbots.
IADIS Int.
J. WWW Internet
2021
,
19
, 17–36.
Available online:
http://www.iadisportal.org/ijwi/
papers/202119102.pdf
(accessed on 25 February 2022).
Electronics
2022
,
11
, 1579
24 of 24
64.
Chaves, A.P.; Egbert, J.; Hocking, T.; Doerry, E.; Gerosa, M.A. Chatbots Language Design:
The Influence of Language Variation on
User Experience with Tourist Assistant Chatbots.
ACM Trans.
Comput.
Hum.
Interact.
2022
,
29
, 13.
[
CrossRef
]
65.
Mehra,
B.
Chatbot
personality
preferences
in
Global
South
urban
English
speaker.
Soc.
Sci.
Humanit.
Open
2021
,
3
,
100131.
[
CrossRef
]
66.
Toader, D.C.; Boca, G.; Toader, R.; M
ă
celaru, M.; Toader, C.; Ighian, D.; R
ă
dulescu, A.T. The Effect of Social Presence and Chatbot
Errors on Trust.
Sustainability
2020
,
12
, 256.
[
CrossRef
]
67.
Schroeder, J.; Schroeder, M. Trusting in Machines:
How Mode of Interaction Affects Willingness to Share Personal Information
with
Machines.
In
Proceedings
of
the
51st
Hawaii
International
Conference
on
System
Sciences,
Waikoloa,
HI,
USA,
3–6
January 2018.
Available online:
https://scholarspace.manoa.hawaii.edu/bitstream/10125/49948/paper0061.pdf
(accessed on 2
March 2022).
68.
Svikhnushina, E.; Pl
ă
cint
ă
, A.; Pu, P. User Expectations of Conversational Chatbots Based on Online Reviews.
In Proceedings of
the DIS’21, Virtual, 28 June–2 July 2021.
[
CrossRef
]
69.
Andrews, P.Y. System personality and persuasion in human-computer dialogue.
ACM Trans.
Interact.
Intell.
Syst.
2012
,
2
, 1–27.
[
CrossRef
]
70.
Cheng, Y.; Jiang, H. How Do AI-driven Chatbots Impact User Experience?
Examining Gratifications, Perceived Privacy Risk,
Satisfaction, Loyalty, and Continued Use.
J. Broadcast.
Electron.
Media
2020
,
64
, 592–614.
[
CrossRef
]
71.
Meli
á
n-Gonz
á
lez, S.; Guti
é
rrez-Taño, D.; Bulchand-Gidumal, J. Predicting the intentions to use chatbots for travel and tourism.
Curr.
Issues Tour.
2021
,
24
, 192–210.
[
CrossRef
]
72.
Svikhnushina, E.; Pu, P. Key Qualities sof Conversational Chatbots—The PEACE Model.
In Proceedings of the IUI
′
21:
Interna-
tional Conference on Intelligent User Interfaces, College Station, TX, USA, 14–17 April 2021.
[
CrossRef
]
73.
Tsekouras, D.; Li, T.; Benbasat, I. Scratch my back and I’ll scratch yours:
The impact of user effort and recommendation agent
effort on perceived recommendation agent quality.
Inf.
Manag.
2022
,
59
, 103571.
[
CrossRef
]
74.
Sonntag, M.; Mehmann, J.; Teitberg, F. AI-based Conversational Agents for Customer Service—A study of Customer Service rep-
resenattive’ Perceptions Using TAM 2.
In Proceedings of the 17th International Conference on Wirstchaftsinformatik, Nuremberg,
Germany, 21–23 February 2022.
Available online:
https://aisel.aisnet.org/cgi/viewcontent.cgi?article=1119&context=wi2022
(accessed on 10 March 2022).
75.
Xu, Y.; Shieh, C.H.; van Esch, P.; Ling, I.L. AI customer service:
Task complexity, problem-solving ability, and usage intention.
Australas.
Mark.
J.
2020
,
28
, 189–199.
[
CrossRef
]
76.
Brüggemeier,
B.;
Lalone,
P.
Perceptions
and
reactions
to
conversational
privacy
initiated
by
a
conversational
user
interface.
Comput.
Speech Lang.
2022
,
71
, 101269.
[
CrossRef
]
77.
Taehyee, U.; Taekyung, K.; Namho, C. How does an Intelligence Chatbot affect Customers Compared with Self-Service Technology
for Sustainable Services?
Sustainability
2020
,
12
, 5119.
[
CrossRef
]
78.
Hildebrand, C.; Bergner, A. Conversational robo advisors as surrogates of trust:
Onboarding experience, firm perception, and
consumer financial decision making.
J. Acad.
Mark.
Sci.
2021
,
49
, 659–676.
[
CrossRef
]
79.
Schuetzler, R.M.; Grimes, G.M.; Giboney, J.S. The effect of conversational agent skill on user behavior during deception.
Comput.
Hum.
Behav.
2019
,
97
, 250–259.
[
CrossRef
]
80.
Stanley, H.Y.B.; Chih-Jen, L.; Shih-Chin, L.T. Toward a Unified Theory of Customer Continuance Model for Financial Technology
Chatbots.
Sensors
2021
,
21
, 5687.
[
CrossRef
]
81.
Presti, L.L.; Maggiore, G.; Marino, V. The role of the chatbot on customer purchase intention:
Towards digital relational sales.
Ital.
J. Mark.
2021
,
2021
, 165–188.
[
CrossRef
]
82.
Popa, I.;
S
,
tefan, S.C.; Olariu, A.A.; Popa,
S
,
.C.; Popa, C.F. Modelling the COVID-19 Pandemic Effects on Employees’ Health and
Performance:
A PLS-SEM Mediation Approach.
Int.
J. Environ.
Res.
Public Health
2022
,
19
, 1865.
[
CrossRef
]
83.
Popa, I.;
S
,
tefan, S.C.; Albu, C.F.; Popa,
S
,
.C.; Vlad, C. The Impact of National Culture on Employees’ Attitudes Toward Heavy
Work Investment:
Comparative Approach Romania vs.
Japan.
Amfiteatru Econ.
2020
,
22
, 1014–1029.
[
CrossRef
]
