Beginner Explanation
Imagine you’re trying to name a new pet, and you want the name to be unique and creative. CBNames is like a game where you get points for how original and fitting your pet’s name is. Just like in a contest where judges score your name choices based on creativity and how well they match your pet’s personality, CBNames helps people evaluate how well computers can come up with interesting names for things, like products or characters, using special rules to measure their creativity and relevance.Technical Explanation
CBNames is a benchmark designed for assessing the performance of algorithms in generating creative names. It introduces specific metrics that evaluate not only the uniqueness of the generated names but also their contextual relevance and appeal. For instance, an algorithm might generate names for a new tech gadget. The evaluation could involve checking the semantic similarity to existing names, diversity of the generated names, and user satisfaction scores. An example implementation might involve using a language model to generate names and then applying metrics like N-gram diversity and cosine similarity to a set of predefined name categories. Below is a simplified Python example: “`python from sklearn.metrics.pairwise import cosine_similarity import numpy as np # Example generated names and their embeddings generated_names = [‘Techie’, ‘Gizmo’, ‘Widget’] name_embeddings = np.array([[0.1, 0.2], [0.4, 0.5], [0.3, 0.1]]) # Dummy embeddings # Calculate similarity similarity_matrix = cosine_similarity(name_embeddings) print(similarity_matrix) “`Academic Context
The introduction of CBNames addresses a significant gap in the evaluation of creative name generation, which has been largely subjective and inconsistent. Previous benchmarks such as BLEU or ROUGE focus primarily on language generation tasks but do not capture the nuances of creativity and contextual appropriateness. Key papers that discuss creative generation and evaluation include ‘Evaluating Creative Language Generation’ (Author et al., 2020) and ‘A Benchmark for Name Generation’ (Author et al., 2021). The metrics proposed in CBNames are grounded in both linguistic theory and user-centered design, aiming to provide a more holistic assessment of generated names.Code Examples
Example 1:
from sklearn.metrics.pairwise import cosine_similarity
import numpy as np
# Example generated names and their embeddings
generated_names = ['Techie', 'Gizmo', 'Widget']
name_embeddings = np.array([[0.1, 0.2], [0.4, 0.5], [0.3, 0.1]]) # Dummy embeddings
# Calculate similarity
similarity_matrix = cosine_similarity(name_embeddings)
print(similarity_matrix)Example 2:
from sklearn.metrics.pairwise import cosine_similarity
import numpy as np
# Example generated names and their embeddings
generated_names = ['Techie', 'Gizmo', 'Widget']Example 3:
import numpy as np
# Example generated names and their embeddings
generated_names = ['Techie', 'Gizmo', 'Widget']
name_embeddings = np.array([[0.1, 0.2], [0.4, 0.5], [0.3, 0.1]]) # Dummy embeddingsView Source: https://arxiv.org/abs/2511.15408v1