AI Controls
Creating precisely what you need with AI requires understanding how to control the AI's behavior. In this guide, we'll explore the various controls available in Chibi and how they affect your results. We'll start with the fundamentals and build up to practical applications.
The Foundation: How AI Generates Text
Before diving into specific controls, it's essential to understand how AI generates text. Modern AI models generate content one token at a time (tokens are roughly equivalent to word parts). For each token, the AI:
Looks at all the tokens that came before
Calculates probabilities for what might come next
Chooses the next token based on these probabilities
Repeats this process until completion
This token-by-token generation process is crucial to understand because each control we'll discuss influences how the AI makes these choices.
Core Controls
Temperature: Balancing Creativity and Consistency
Temperature is perhaps the most important control, affecting how the AI chooses between possible next tokens.
What It Does:
Low temperature (0.0 - 0.3): The AI consistently picks the most probable tokens
Medium temperature (0.4 - 0.7): The AI balances between probability and variety
High temperature (0.8 - 1.0): The AI is more likely to pick less probable tokens
Best Used For:
Low: Factual writing, technical documentation, structured formats
Medium: General content, blog posts, creative but focused writing
High: Brainstorming, creative writing, generating diverse ideas
Practical Example: When asked "What's a good breakfast?", the AI might respond:
Low temperature: "A balanced breakfast consists of protein, whole grains, and fruit."
High temperature: "Start your day with a vibrant symphony of golden honey-drizzled granola, paired with tangy berry compote and creamy Greek yogurt!"
TopP (Nucleus Sampling): Controlling Token Selection Range
While temperature affects how the AI chooses tokens, TopP determines which tokens it considers in the first place.
What It Does:
Low TopP (0.1 - 0.3): Considers only the most probable tokens
Medium TopP (0.4 - 0.7): Considers a moderate range of tokens
High TopP (0.8 - 1.0): Considers a wider range of possible tokens
Best Used For:
Low: When you need very focused, precise outputs
Medium: General purpose content creation
High: When you want the AI to consider less common but potentially interesting options
Important Note: TopP and temperature work together. A high temperature with low TopP still limits creativity because the AI has fewer tokens to choose from.
Advanced Controls
Understanding Penalties
Chibi offers two distinct penalties that help control repetition in different ways. Both can range from -2.0 to 2.0, with 0.0 as the default (no effect).
Frequency Penalty: Managing Token Frequency
The frequency penalty looks at how often tokens appear in the input and adjusts their probability in the output accordingly. The more a token appears, the stronger the effect.
How It Works:
Positive values (0.1 to 2.0): Discourage the use of tokens based on how frequently they appear in the input. The more a token appears, the less likely it becomes.
Zero (0.0): No effect on token frequency
Negative values (-2.0 to -0.1): Encourage the reuse of tokens based on their frequency in the input. The more a token appears, the more likely it becomes.
Example: Let's say your input contains the word "technology" several times. With a:
Positive frequency penalty: Each appearance of "technology" makes it increasingly less likely to be used again
Negative frequency penalty: Each appearance of "technology" makes it increasingly more likely to be used again
Zero frequency penalty: The frequency of "technology" in the input doesn't affect its probability in the output
Presence Penalty (Repeat Penalty): Managing Token Presence
The presence penalty looks only at whether a token has appeared in the input at all, regardless of how many times. It applies a flat adjustment based on presence alone.
How It Works:
Positive values (0.1 to 2.0): Discourage the reuse of any token that appears in the input, regardless of how often it appears
Zero (0.0): No effect on token reuse
Negative values (-2.0 to -0.1): Encourage the reuse of tokens that appear in the input
Example: If your input contains the words "artificial" and "intelligence" once each, with a:
Positive presence penalty: Both words become less likely to be used again, by the same amount
Negative presence penalty: Both words become more likely to be used again, by the same amount
Zero presence penalty: The presence of these words doesn't affect their probability in the output
When to Use Penalties
It's best to start without penalties (0.0) and only add them when you observe specific issues:
Use frequency penalty when:
You notice certain words or phrases being overused proportional to their appearance in the input
You want to encourage (negative values) or discourage (positive values) the AI to build upon patterns in the input
Use presence penalty when:
You want to broadly encourage (negative values) or discourage (positive values) the AI from reusing any tokens from the input
You need a simpler approach to managing repetition that doesn't scale with frequency
Best Practices for Penalties
Start without penalties (0.0 for both)
Observe the AI's natural output with your prompt and context
If you notice issues with repetition:
For scaling repetition issues (words being used more as they appear more), try small adjustments to frequency penalty first (try 0.1 or 0.2)
For general repetition issues (any reuse of input tokens), try small adjustments to presence penalty first (try 0.1 or 0.2)
Use negative values cautiously, as they can lead to excessive repetition
Remember: These penalties are advanced tools. Most use cases work well with default values (0.0). Only adjust them when you have a specific repetition issue to solve.
Practical Recommendations
For Different Content Types:
Technical Writing:
Temperature: 0.1 - 0.3
TopP: 0.4 - 0.7
Repeat Penalty: 0.0 - 0.1
Frequency Penalty: 0.0 - 0.1
Blog Posts:
Temperature: 0.4 - 0.6
TopP: 0.6 - 0.9
Repeat Penalty: 0.0 - 0.2
Frequency Penalty: 0.0 - 0.2
Creative Writing:
Temperature: 0.7 - 0.9
TopP: 0.7 - 1
Repeat Penalty: 0.0 - 0.3
Frequency Penalty: 0.0 - 0.3
Common Pitfalls to Avoid
Over-constraining the AI:
Using low temperature AND low TopP
Setting both penalties too high
Result: Stilted, unnatural text
Under-constraining the AI:
Using high temperature AND high TopP
Setting no penalties
Result: Potentially incoherent or rambling text
Mismatched Settings:
High temperature with low TopP
Very high penalties with low temperature
Result: Conflicting goals leading to poor output
Best Practices for Getting Started
Start with moderate settings:
Temperature: 0.5
TopP: 0.98
Repeat Penalty: 0.0
Frequency Penalty: 0.0
Adjust one control at a time and observe the changes
Keep notes about which combinations work well for different types of content
Remember that different AI models may respond differently to these controls
Think of these settings like this:
Write naturally, using your full vocabulary (TopP: 0.98)
But be thoughtful and somewhat consistent in your word choices (Temperature: 0.5)
Without worrying about repetition rules (Penalties: 0.0)
For getting started, these settings are great because:
It lets you see the AI's natural capabilities
Makes it easier to understand how temperature affects the output
Produces more consistent and predictable results
Helps you establish a good baseline before experimenting with penalties
Conclusion
Understanding AI controls is crucial for getting the most out of Chibi. Start with the recommended settings and experiment gradually to find what works best for your specific needs. Remember that these controls work together as a system, and finding the right balance is key to achieving your desired results.
The beauty of Chibi is that once you find settings that work well for a particular type of content, you can save them in roles or actions for consistent results. Take time to experiment and find the combinations that work best for your specific use cases.