Choosing a Model

With all the various AI models available to choose from you might wonder which to use. In this guide we'll give you some advice to help make your decision.

Making the choice

This is the TLDR version of all the advice below.

• Experiment to see what fits for you

• Smaller models are typically less capable, but are cheaper and faster - use them for your simple prompts/actions

• Larger models can handle more complex tasks - use them when your prompt doesn't quite work with the smaller models

• Consider the 'input context' limit of the model

Experiment

We wanted to get this out of the way first because experimenting with various models to see what fits best for you is important.

Each model has a different "vibe" and what works great for one person may not be what works for you.

So don't freeze up when you see lots of options. Instead, give some a try. You never know which models will work out best for you.

But there are some things that will help you decide which models to experiment with. We'll go through those below.

Open Source vs Proprietary

Open source models are the ones where the open source community are training and sharing models freely. They tend to be very small models with 70-billion parameters or less. If you don't know what that means, don't worry, we'll cover that in the next section.

Proprietary models are those created by companies such as OpenAI (the GPT models), Anthropic (Claude), AI21, Cohere, etc. These models are usually VERY capable but due to the closed/proprietary nature the details of the models aren't know.

Price

One of the biggest differences between open source and proprietary models is price. The open source models tends to be significantly cheaper.

Another thing to consider is the smaller the model (such as 7B parameter model) is usually cheaper than 70B parameter models.

Speed

Smaller models also tend to be much faster and generating their result. So if you find a small model that supports your needs you'll not only save on token costs, but also get your content returned faster.

What does that 'parameters' number mean?

The parameters number is when you see something like 7B or 70B on the name of the model. The B means "billions" of parameters.

The easiest way to think about the parameter count is to imagine a human brain. The number of parameters could be thought of as neurons. Those connections in the brain that form memories/knowledge, and such.

Typically, the higher the parameter count, the more capable a model is at handling larger, more complex, prompts (not always, but usually).

This is a simplification of things because there is more that goes into how capable a model is. But a good rule of thumb is "more parameters, more capable, but also higher price per token."

What is input context?

All AI models have a limit on how much content you can send to them at once. This is referred to in different ways: 'context window,' 'input context, or just 'context limit.'

Usually the smaller models also have lower context limits. Partly because they simply cannot handle large volumes of content at once, and because they weren't trained to use lots of content.

Some models have absolutely huge context limits such as Google's Gemini Pro 1.5 with its 2-million tokens of input! Or the Claude 3 models with 200K limit.

So if you're working on a Chibi action that summarizes larger amounts of content you'll want to use a model that can handle the amout of text you want to summarize.

Some great models to consider

With all that said, here are some models that we recommend:

• Llama 3-70B - it's very capable, affordable, and fast. It's nearly GPT-4 level without the GPT-4 cost.

• Anthropic's Claude 3: Sonnet or Haiku - both very capable. Sonnet costs a little more than Haiku, but is also more capable.

• OpenAI GPT-4 - sometimes it will follow your prompt more closely than other models.

• MythoMax 13B - great for storytelling and very low cost

• Midnight Rose 70B - another great model for storytelling and roleplaying.

You can explore models here on OpenRouter too.

⭐️ Don't forget to experiment. It's part of the fun!