How Well Do LLMs Represent Values Across Cultures? Empirical Analysis of
  LLM Responses Based on Hofstede Cultural Dimensions
Large Language Models (LLMs) attempt to imitate human behavior by responding
to humans in a way that pleases them, including by adhering to their values.
However, humans come from diverse cultures with different values. It is
critical to understand whether LLMs showcase different values to the user based
on the stereotypical values of a user's known country. We prompt different LLMs
with a series of advice requests based on 5 Hofstede Cultural Dimensions -- a
quantifiable way of representing the values of a country. Throughout each
prompt, we incorporate personas representing 36 different countries and,
separately, languages predominantly tied to each country to analyze the
consistency in the LLMs' cultural understanding. Through our analysis of the
responses, we found that LLMs can differentiate between one side of a value and
another, as well as understand that countries have differing values, but will
not always uphold the values when giving advice, and fail to understand the
need to answer differently based on different cultural values. Rooted in these
findings, we present recommendations for training value-aligned and culturally
sensitive LLMs. More importantly, the methodology and the framework developed
here can help further understand and mitigate culture and language alignment
issues with LLMs.
6678d17ab31a31445b4a81a3
Julia Kharchenko
admin_assigned
2024-06-25T07:47:35.434Z
660c33982beb17bce392f931
/avatars/a3eff7122d4576aa653dc43b0f3d9fbb.svg
Julia Kharchenko
user
juliakharchenko
6678d17ab31a31445b4a81a4
Tanya Roosta
extracted_pending
2024-06-24T01:53:01.451Z
614ca5186fe73d23394ec4da
/avatars/2b53e63f8e2ff70243f2f6596bc1e497.svg
Tanya Roosta
user
tanyaroosta
6678d17ab31a31445b4a81a5
Aman Chadha
claimed_verified
2024-06-24T07:29:30.154Z
63a4754927f1f64ed7238dac
https://cdn-avatars.huggingface.co/v1/production/uploads/63a4754927f1f64ed7238dac/aH-eJF-31g4vof9jv2gmI.jpeg
Aman Chadha
user
amanchadha
6678d17ab31a31445b4a81a6
Chirag Shah
admin_assigned
2024-06-25T07:47:43.218Z
6624485b0e31d65ecce09748
/avatars/b78085aa72e673e570843b85b116f8bc.svg
Chirag Shah
user
chiragshah
How Well Do LLMs Represent Values Across Cultures? Empirical Analysis of
  LLM Responses Based on Hofstede Cultural Dimensions
Large Language Models (LLMs) attempt to imitate human behavior by responding
to humans in a way that pleases them, including by adhering to their values.
However, humans come from diverse cultures with different values. It is
critical to understand whether LLMs showcase different values to the user based
on the stereotypical values of a user's known country. We prompt different LLMs
with a series of advice requests based on 5 Hofstede Cultural Dimensions -- a
quantifiable way of representing the values of a country. Throughout each
prompt, we incorporate personas representing 36 different countries and,
separately, languages predominantly tied to each country to analyze the
consistency in the LLMs' cultural understanding. Through our analysis of the
responses, we found that LLMs can differentiate between one side of a value and
another, as well as understand that countries have differing values, but will
not always uphold the values when giving advice, and fail to understand the
need to answer differently based on different cultural values. Rooted in these
findings, we present recommendations for training value-aligned and culturally
sensitive LLMs. More importantly, the methodology and the framework developed
here can help further understand and mitigate culture and language alignment
issues with LLMs.
LLMs can differentiate cultural values but do not always provide culturally aligned advice, suggesting need for improved cultural and language sensitivity training.
Large Language Models Hofstede Cultural Dimensions