Is there a more effective way to build a language than a word frequency list?

Question

I'm in the process of creating my own language, and so far, I've built a basic vocabulary set of about ~200 words. From here, however, I'm somewhat unsure as to what vocabulary I should work on adding first. Obviously creating a language with the vocabulary of a full natural language, with hundreds of thousands of words, is both improbable and impractical. However, I do want to be able to express a variety of thoughts and ideas in my language, as we would in any other natural language, so to get this effect, I've been using Wordcount, which tracks the frequency of English words and ranks them.

According to Nation & Waring (1997),

• Most frequent 1000 words: 72.0%
• Most frequent 2000 words: 79.7%
• Most frequent 3000 words: 84.0%
• Most frequent 4000 words: 86.8%

As you can see, knowing the first 2000 words of a language accounts for almost 80% of use in everyday communication. Because of these statistics, I'm inclined to keep using word frequency lists. However, I'm not sure if this is the most effective way to go about it.

Is there a a more effective way to build the vocabulary of a fictional language than a word frequency list? (I don't want Just go through the dictionary! as an answer, by the way.)

Answer 1

I don't have a reference but some philologists and linguists have speculated that you could survive (explain who you are, ask for employment and food, etc.) in a foreign land with as few as 500 words. Henry C. Fenn, author of several language texts, felt that a vocabulary of 5000 words was sufficient to support learning new words by context, e.g. by conversing with natives and reading newspapers. In my own experience as a linguist and translator I found 5000 generic words plus about 2000 topical words (those popular for only a few years) sufficient for keeping up with current events. This gives you some idea of the vocabulary size you should be considering.

I agree that generating a vocabulary by simply making up alternates to the 5000 to 7000 most popular words in your native language is tedious and slow. Worse, it is not fun. Even more worse, the resulting language is at an elevated risk of being insipid and possibly silly.

I suggest that you first create a list of word elements, and then compose most of your vocabulary words by altering and conjoining these elements. If you read up on Grimm's Law, and how Proto-Indo-European (PIE) was reconstructed, you'll see what I mean.

Web-search "proto indo european word list" and build up a list of element meanings. Throw away the PIE words and substitute your own. You'll probably only need to make about 300 word roots.

Make up some rules about combining roots into words. You don't need to be perfect or complete because you are aping a process that takes place over millennia, during which the rules would change.

Make up some rules about how sounds might change over time, and apply them consistently across all words. For example: after generating 1000 or 1500 words, change all 'P' sounds in your element list to 'B' and all 'T' to 'D'.

Now -- here's how you relieve the tedium and labor, which is the point of all this: Since you are confident that you have a rational method for producing believable words, don't make up all 7000 words at once. Start with a few hundred, and after that, create new dictionary entries as you need them.

Whenever you need a word, you look it up in your dictionary. If it's not there, you generate it and add it to the dictionary at that time. Since you already have lots of rules for forming words, making up the new one will be quick.

If you don't want to be the sole authority on word formation, you can roll dice or write a randomizing computer program. If the language is being used in a role-playing game, the rule could be "First player to open the dictionary gets to generate the new word."

What about the 200 words you already have? Well consider this. English, although a PIE group language, is actually descended from two distinct branches, the Teutonic and the Romance. (That's why English spelling is so hard. Almost half of our words come from low German, Anglic, Danish, etc. and present a Germanic spelling, while almost half come from Latin, French, and Norman, and present an Old French spelling.)

So, if you have any words from your original 200, that you want to keep, that cannot be rationally derived from your element list, simply declare that they arose from a separate branch or even a separate proto-language.

I don't see how this could fail to be fun.

Answer 2

My recommendation would be to conduct a mock conversation in your head between two speakers who are typical of the culture in question.

Some things to keep in mind:

• What will the speakers of the language likely talk about? For example, some topics might be used more in a medieval setting (e.g. "ax") than in an outer space setting. Pick a conversation dealing with a common thread of interest.
• What speech patterns are the speakers likely to fall into? For instance, Klingons often don't use greetings but get straight to the point, while other species use them a lot. "Hello" may not be one of the most common words, but it's going to be essential.

Using this envision, two typical speakers talking about a relatively common subject. Go through it in your head in your native language, and figure out what words you'll need. Many dialogues will have some common features, even if the precise content differs. Again, there will be common speech patterns and shared phrasing.

The reason I would choose a conversation over a word list is that the most common words in English may not be the most necessary words in this language. Various factors will influence which words are essential for every conversation (which is why I gave greetings as an example), and which ones aren't. Frequency is not always the best determination of importance.

Answer 3

A man called Mark Rosenfelder has written three well-regarded books on creating a conlang. The one most relevant to your query is The Conlanger's Lexipedia. Among other things it contains a "Fantasy Frequency Wordlist, a 1500-item list of the most common roots from a 1.1-million-word corpus of fantasy and science fiction". Equally important, there is also a material about how to make your word choices and definitions believably different from English and/or human languages in general.

I confess that I have not put in the hard work necessary to do more than play around the edges of creating a conlang, but I found all three books very interesting to read. The other two are The Language Construction Kit and Advanced Language Construction, both of which also have sections dealing with vocabulary building. If you wanted a lighter book, Holly Lisle's Create a Language Clinic is an accessible introduction to the subject. It's much less scientific than Rosenfelder's books but gets you started right now.

In my experience the paper versions of conlanging books are easier to work from than the e-book versions.

Answer 4

Natural Semantic Metalanguage

The Natural Semantic Metalanguage (NSM) is a mini-language of sorts, for which any other language (at least the ones they have studied, which I understand to be numerous) can be broken down into. For example, a translation of the English word "happy"

someone X is happy (at this time) -translates to->

• someone X thinks like this at this time:
  • "many good things are happening to me as I want
  • I can do many things now as I want
  • this is good"
• because of this, this someone feels something good at this time
  • like someone can feel when they think like this

It contains 56 primitive "words" or primes, from which phrases like the above can be made. Since it is not a full language, these "words" weren't given spellings or anything. It also doesn't have a grammar. If you provide words in your language that correspond to these, you will be able to say anything you need, although it might be long. This is good though if your language is missing some words.

Word List
Example Translations

Again, I wouldn't base your language entirely off of this, but its a good start and can fill in any holes until you fill them.

Answer 5

If all words in the language match 1-to-1 to English words, it's basically simply a code for English.

Pitjantjatjara for example doesn't have a word that means what the English word brother means.

Even European languages differ in the contexts the have. If I want to translate the German word Wahrscheinlichkeit Polish offers me one choice with prawdopodobieństwo according to Leo. On the other hand English offers me 8 choices: probability, verisimilitude, chance, plausibility, presumption, feasibility and likeliness.

English allows to make much more distinctions then Polish when talking about probabilities.

On the other hand the English languages uses words like to feel or to be in lot's of different ways. To be essentially serves the purpose of four terms of the Natural Semantic Metalanguage location, existence, specification and possession.

Letting 4 of the 56 most important concepts of a language be represented by the same word seems very wasteful and yet English works fine. This means that your language can also reuse words if you don't have enough words.

Depending on the purpose of your language you don't have to add all the word at the start. When you find that you currently can't talk about something that would be important you can just add new words.

Answer 6

Don't do a general purpose word list, if you are creating your own language, you are probably creating that for some purpose (I am assuming for some book, game or some fictional work). So first create words for sentences that are actually used in your work (that's it, make your work worth, do not spend time inventing stuff that probably you will not use).

It is pointless in example making most common 5000 words if you are writing a story of a wizard working in his magic tower using technical names.

Also be creative, and re-use existing words that are no longer in use, to regular reader those words will just look like a new language, but at least some of them will have a meaning for an above average person. As a programmer I find in fact there are many informatics-related terms that could be of use in real life, but unluckily most non-pogrammers don't know the meaning of those words (in example: the concept of "cache locality" and efficiency explains why disorderly people is more productive, when I see other's people mess I think "it is just their cache" , but to regular people a mess is just a mess).

Also invent some etymology, it should be easy to use few words to make new words out of that, that gives your language a story and make it more realistic. The language depends very much on how its world behave, so be sure to have a very realistic world, try to figure out possible situations on that world, take example from authors that already tried to do that and find out fallacies and strong points in their languages.

the language evolve in the context, for example for most farm's animals we have different names for babe, male and female versions, while for animals outside farms we have just a unique name, that's because of our past working in farms).

Answer 7

I apologize in advance because I feel this would be better suited as a comment, but I don't have the requisite reputation.

I think all the other answers are great, but I just want to add another consideration that I used myself on my last conlang and found really helpful. If you consider semantic domains in vocabulary generation, you'll much more easily avoid remapping English (or whatever your preferred language is). I used to have a PDF that had several dozen semantic domain webs that covered lots of basic vocabulary and prompted more obscure words too, but the link's dead now and for the life of me I can't find a good replacement through some cursory googling. But I'm sure if you do some searching of your own you can find some good basic examples to look at.

Essentially what this entails is you have a web of related concepts, and different languages will assign different lexemes to different groupings of these concepts. For example, in English we have: cloud, fog, mist, steam, smoke; and without bothering to look up their etymologies, those words all look unrelated. Looking at concept webs like this might help you think about whether you want one word to cover "cloud" and "fog" but have two different words for steam from boiling water and steam from warm water (such as at a hot spring in the winter).

I wish I had ready-made semantic domain webs or maps to link you to, but I'm finding it hard to find anything better than generic examples. As a supplemental tool, though, I think it's something that shouldn't be too hard to make on your own in your native language just to use as a jumping-off point for vocab generation. I don't think you'll hit 10,000 words like this, but I can personally attest that it can improve your generation rate significantly for your first several hundred words once your get yourself in the habit of immediately considering all the conceptually related lexemes for each basic word. You might collapse some into one word and add extra distinctions for others. It is kind of mind-boggling how many different words humans employ for things that are barely different. (Of course, if you make a hyper-polysynthetic language, maybe you just need one root per domain!)

Answer 8

When inventing a language you are basically a language learner (of your own language). You have to figure out which words are needed to make use of the language. Therefore I suggest that you don't reinvent the wheel but have a look what others did already in this field.

Your world is the typical late Roman/early Medieval style? Then a Latin or Greek learner's vocabulary would be perfectly what you're looking for: It contains normally between 1500 to 5000 (depending on the level) very common words ranging from daily life (e.g. food names, trading, etc.), political life (e.g. "career", "bribe", etc.), religious life (words for "temple", "rite", etc.) to military life (e.g. words for weaponry and other warfare equipment).

Your world is rather steampunk? Then a late 19th century French-English or English-French learner's vocabulary might cover what you need. These are hard to get, I know, but chances are that http://www.archive.org or http://books.google.com host some scans. Otherwise check antiquarian shops (all types: books, furniture, etc.). Such books are normally of rather little (monetary) value, so they shouldn't cost you too much. I bought a 19th century French dictionary for as little as 8 CHF (ca. 8.50 USD).

If your world is sci-fi, then a modern-day language learner's vocabulary would suffice. It contains daily life termns, technological terms, etc.

That method helped me a lot creating my first few languages. Learners' vocabularies (and grammars) always cover perfectly the basics to make a language usable.

Answer 9

mostly a summary of other answers:

1. invoke the yagni principle and only invent words as you need them. simply add them to the dictionary as you invent them for your story dialog.
2. you will need to invent both vocabulary (words) and grammar (e.g. sentence structure) in order to create a truly unique language (rather than just an english word codebook).
3. you will probably want to invent roots, conjugations and declensions (e.g. prefixes, vowel-shifts, suffixes) in order to make a realistic and coherent vocabulary. a good introduction to linguistics will give you an idea of what is possible.
4. in addition to the language, you will probably want to think about the writing system. critically, phonetic vs non-phonetic and ligatures.
5. seek the advice of someone who has actually accomplished what you are trying to do. e.g. tolkien wrote a book (a secret vice) about inventing languages.

Answer 10

As usual for this type of questions, the usual disclaimer. The way you would go about creating a language depends severely on the purpose you want to put it to. If your goal is to just put a couple of alien-sounding words in a work of fiction, you can just invent them as you go along. If you want a conlang to communicate in, you can go by the frequency list - it would be more of substitution code then a language.

If, however, your aim is to create a language that is semi-independent from the work of fiction you write, I'd propose to go another way about it. I will draw now on both my expirience of working with Natural Language Processing and hobby interest in Tolkien's languages.

First, you need phonology and grammar, in whatever order. If you are creating a lifelike language from ground up, you are going to work with roots, not with individual words. So, before you go there, you need to figure out, what phonemes you have in your language, what are the phonetic rules working in it. You need to understand which speechparts you have and the way the root can be converted to another speechpart.

Roots come next. I would recommend to use not frequency list, but Swadesh list. The words on the list would be most archaic - and that means a lot of things. First, going through this list, you can make corrections that would be important for your setting and the people that speak this language. Maybe they have no gender roles and didn't have them since the very beginning of the species - then they won'd have separate roots for man and woman. Another important thing of the archaic roots would be that they will be mostly irregular in conjugation and declension.

When you will be past those steps, having the basic grammatical framework in place, phonetical laws and basic (Swadesh, not frequency) vocabulary, you will have the feel of your language, it's rhythm and cadence, understand the way the phrases are built, etc. Now you can go on filling the gaps in your dictionary as neccessary by writing texts in that language.

A nice bonus to a way of constructing the language this way is that you can apply the laws of glottochronology to it to create related languages of different degrees of separation - that is more or less what Tolkien did with various branches of elvish languages.

Answer 11

One possibility is to take a real language completely foreign to you (that you don't know), then learn it. (e.g. those learn-language-on-tape type things)

However, every word that you learn from that language, substitute your own word.

A few drawbacks I know, e.g.

1. You won't get a proper etymology and therefore words that "should" sound similar won't. (Though you could modify with respect to your world's history & culture)

2. You'll have to live with the same sentence structure as the "factual" language

3. You may end up just coming up with ways of asking directions to the beach ;)

Answer 12

I have a better idea for you. How about you add a word when its needed! So say your writing a letter to your girlfriend, and you need a word for love, make one then! That way you don't have to worry about having to create thousands of words on the spot and not use them ever again. Because remember the most frequent words aren't your most frequent words!