Demo

Explore the data in AI Law of Jante

Hypothesis

Can LLM’s be used for research? how LLMs can help us speed up research by quickly summarizing the perception (at least on the Internet) of something complicated as culture.

Learnings

Data

Prompting

Design

Design Diagram

Generating the Law of Jante (Prompt Engineering)

Generating the Law of Jante for each country

Costs of creating the Prompts with GPT 4

Prompt I am using to generate for each country

Generate a list of ten cultural rules in the style of the Jante Law for the country {country}.

Use the same style as this example for Denmark:
1. Humility and Modesty: You're not to think you are anything special.
2. Equality and Uniformity: You're not to think you are as good as we are.
3. Collective Wisdom over Individual Knowledge: You're not to think you are smarter than we are.
4. Egalitarianism: You're not to imagine yourself better than we are.
5. Respect for Collective Experience: You're not to think you know more than we do.
6. Community Focus over Individual Importance: You're not to think you are more important than we are.
7. Self-effacement and Anti-Exceptionalism: You're not to think you are good at anything.
8. Solidarity and Respect for Others: You're not to laugh at us.
9. Self-reliance within the Community: You're not to think anyone cares about you.
10. Collective Learning and Shared Knowledge: You're not to think you can teach us anything.

Matching rules for comparison (Embeddings)

Finding a way to compare countries’ rules

Classifier of rule similarity (Prompt Engineering)

Prompt

Please rate these two cultural rules in terms of how similar they are with a score of -5 to +5
(-5 means they are the opposite of each other, 0 means they are unrelated/neutral to each other,
and +5 means they are basically the same)

Rule 1: {rule1} 
Rule 2: {rule2}

Please output in the following format:
Rationale: explain the reasoning around scoring
Score: (value between -5 and 5)
PLEASE MAKE SURE YOU ALWAYS OUTPUT THE RATIONALE AND THE NUMERIC SCORE
GroupPercentage Missing GPT-3 Score
Prompt without CAPS29.5%
Prompt with CAPS23.4%

Second Classifier of rule similarity (Prompt Engineering)

Prompt

Here are examples of rationales, that describe how two cultural rules compare with each other and their respective score. Scores range from +5 (if they completely agree with each other) to -5 (if they are in complete disagreement with each other). A score of 0 is used for neutral scoring, where the rules are not related to each other at all. A positive score indicates a degree of agreement or similarity between the rules, with higher scores indicating greater similarity. A negative score indicates disagreement or contrast between the rules, with lower scores indicating greater disparity.

Example 1:
Rationale: Rule 1 emphasizes the importance of the community and discourages individualism, while Rule 2 promotes individualism and self-reliance. These two rules seem to be in direct opposition to each other, as one prioritizes the collective and the other emphasizes the individual.
Score: -5

Example 2:
Rationale: Both Rule 1 and Rule 2 emphasize the importance of respecting others and not imposing one's beliefs on others. However, Rule 1 specifically focuses on family values, while Rule 2 pertains to political views. While these two rules may have some overlap in terms of promoting respect and avoiding imposition, they are not directly related to each other.
Score: 0

Example 3:
Rationale: Both Rule 1 and Rule 2 emphasize the importance of respect, but they focus on different aspects. Rule 1 highlights the need for religious tolerance and discourages the belief that one's own faith is superior to others. On the other hand, Rule 2 emphasizes the importance of preserving and not exploiting natural resources. While they address different domains (religion and nature), they both promote the idea of respecting something beyond oneself.
Score: 3

Example 4:
Rationale: The two cultural rules are identical in their wording and meaning. Both emphasize the importance of respecting and not considering oneself wiser than elders. Since they are exactly the same, they should receive the highest similarity score.
Score: 5

Example 5:
Rationale: The two cultural rules seem to have conflicting messages. Rule 1 emphasizes humility and modesty, suggesting that individuals should not think they are superior to others. On the other hand, Rule 2 emphasizes the importance of appearance, suggesting that individuals should not neglect their appearance. While both rules address individual behavior and attitudes, they seem to have different focuses and priorities.
Score: -3

Now, please rate the following rationale in terms of similarity, considering the definitions of scoring:

Rationale: {rationale}

PLEASE JUST OUTPUT THE NUMERIC SCORE - JUST THE NUMBER PLEASE

Country Similarity Score Algorithm

def normalize_gpt3_score(score):
   """Normalize the GPT-3 score from -5 to +5 to a scale of 0 to 1."""
   return (score + 5) / 10 if score is not None else None

Similarity Score Distribution (original algorithm)

def normalize_gpt3_score_v2(gpt3_score,gpt3_score_new, similarity_score):
  score = gpt3_score_new
  if gpt3_score <0 and gpt3_score_new>=0:
    score = gpt3_score
  if gpt3_score == 0:
    score = 0
  if(score >0):
    score = (score + 5) * similarity_score / 10
  elif score < 0:
    score /=10
  return score

Similarity Score Distribution (current algorithm)