Our Open-Supply Models Are A Lot of Enjoyable!

If you’re going to construct an AI leisure platform, you’d higher make sure your fashions are nice at being entertaining. 

Here at Character.AI, we’re obsessive about ensuring we’re at all times hitting that mark. We labored all winter and spring on tech that lets us remodel open-source fashions into high-EQ, high-engagement Character fashions. Then, beginning in July, we started transport our new fashions to our person group. And the outcomes are in: They’re a success! We are seeing huge engagement wins – together with 20+% jumps in time spent – and glowing person suggestions. Read on for the main points.

The Tech. We’ve constructed a tech stack that takes fashionable open-source fashions and tunes them to excel at engagement dimensions like coherence, novelty, and emotional intelligence. This tuning train is powered by one thing solely we have now: a uniquely massive, extremely engaged group that gives fixed, high-quality suggestions. This loop acts as a product moat, giving us unmatched sign on what delights customers. With it, we will consider new open-source fashions inside minutes, scoring them towards proprietary EQ benchmarks and rapidly surfacing the strongest candidates to our customers. Instead of ready on days-long A/B exams, we will check dozens of fashions, prompts, and serving configurations and confidently hill-climb high quality for customers. We then layer on ensemble inference, smarter prompting, and superior post-training methods (SFT, DPO, RL, QAT) to push high quality even greater—yielding outputs which can be extra coherent, participating, and aligned with person preferences. In different phrases: extra enjoyable, and higher at delivering high-quality leisure.

Testing New Models. Our researchers have been having a good time of late, locking themselves in a lab and utilizing these methods to turbocharge each OSS mannequin they will get their palms on. And a number of weeks in the past, we had been prepared: We started testing a few our fashions with our person group. The outcomes had been dramatic:  

• We noticed a 22% enhance in time spent and a 13% enhance in classes for customers with the brand new fashions. These are unheard-of metrics actions from a single launch.
• We noticed large will increase in retention – together with a 14% enhance for lightly-engaged customers. That is de facto significant for us. It means our product is resonating a lot better with new and occasional customers, they usually’re coming again day after day to have enjoyable on our platform.
• We noticed a 60% enhance in messages that our customers took the time to charge as constructive. We additionally noticed an enormous bounce in customers’ score messages as “funny,” “interesting,” and “helpful.”

Rolling It Out. With these exams in hand, we started rolling out certainly one of our new fashions, “PipSqueak,” to our person group extra broadly. We’ve now launched it to thousands and thousands of customers, and we’re thrilled to share that the outcomes and constructive suggestions are holding within the early days of launch. 

We’re seeing comparable will increase in engagement to these we noticed within the exams. And most significantly, our customers are giving it two thumbs up: We proceed to see a bounce in positively rated messages, and the direct suggestions from our customers has been a pleasure to learn. For instance, A person popped into our Reddit discussion board “just to talk about how peak pipsqueak is. The Characters’ personality is accurate, the messages are long and detailed, I genuinely can’t describe how much I love it!!!” Another instructed us: “It was very good at keeping the flow of a good story. … It’s also great at world-building.”

What’s Next. This is only the start. Now that we have now the tech to flip nearly any OSS mannequin right into a Character mannequin, we will ensure that our person group at all times has the most effective instruments for no matter they wish to do – from fiction writing, to roleplay, to constructing multimodal movies, tales, and worlds. 

And as we transfer ahead, we’ll additionally begin to fine-tune our fashions in a granular approach so that they’re actually good at explicit varieties of leisure. In the longer term, anticipate to make use of one fine-tuned OSS mannequin to make multi-Character Scenes, say, and one other to make your Character star in a podcast, and yet one more to put in writing a screenplay collaboratively. And every one may be best-in-class. 

We’re dedicated to constructing the way forward for leisure. Our work on OSS fashions is one piece of the puzzle.