Charisma.ai, delivery tools to power the metaverse

Charisma is a storytelling engine that is creating the next generation of interactive stories for any platform - mobile, online and VR. As CEO Guy explains, the team are on a mission to push the boundaries of what artificial intelligence can do with this ‘toolkit for creatives’, which can be used to inspire ideas, accelerate development, and create new forms of entertainment. 

“Our grand vision is to pull together best-in-class technology in a wide variety of domains to power characters in the metaverse and beyond. That includes lip-sync, text-to-speech, natural language processing tech and generative systems like GPT-3. By working together, we hope to delight, excite and engage players for years to come.” 

With integrations into Unreal Engine, Unity, and Decentraland, (and JavaScript, React, React Native, and Python SDKs), the Charisma team are ensuring they are taking the tools directly to where the creatives are already playing.

They are pushing the boundaries of immersive storytelling with a hybrid skill base of creativity and technology.

The platform, which has been built up through writers’ labs, immersive projects, and AI R&D, includes an AI-powered dialogue engine where creatives can build virtual worlds wherein players interact with characters to experience magical (and often surreal!) conversations – see their experimental GPT-3-powered interactive dream landscape Electric Sheep as an example.

Since launching the Charisma platform, the team have collaborated with games studios like Epic Games (Metahumans) and media organisations like WarnerMedia (Steppenwolf) and NBCUniversal. 

You can sign up to the wishlist on Steam to be among the first to experience their most recent project The Kraken Wakes – an immersive alien attack on the oceans of Planet Earth. This exclusive adaptation of John Wyndham’s epic sci-fi/horror novel is a ground-breaking game where you get to speak to the characters and they speak back, voice-to-voice, placing you at the heart of the story.

So... what exactly is the metaverse?

This is a favourite topic and not one that is easy to sum up in a paragraph. However the fundamental takeaway is that it's still evolving. As Guy explains;

"The first VR consumer applications have been focused on entertainment or training environments and are primarily extensions of existing highly scripted experiences. However Charisma see the metaverse as being much more about an always-on convergence of immersive visual experiences like VR and AR with our current digital economy and social interactions that the internet underpins."

And this appears to underpin the whole Charisma approach; that the real dramatic potential will evolve as the advanced deep learning NLP models like GPT become increasingly responsive, intuitive and intelligent – some might even say ‘sentient like’, all based on strong principles of storytelling, narrative and entertainment.

On the question of sentience Guy laughs and says….

"Unlike chatbot platforms, Charisma uses the language of storytelling, with built-in features like emotion, memory, scenes and subplots to bring your projects to life. Sometimes this can start to feel very close to sentience, especially if the data that the model is trained on has been trained on aspects of life that is very like your own, and is in the hands of a good storyteller. A well-written horoscope or magic trick sometimes feel like they 'understand' us, but this is not quite sentience. The definition of sentiences is at the core of human philosophy, so it's funny to try to claim it for a machine."

How does Pipeline.ai support this type of creativity?

The key to real immersion is the extent that you can suspend disbelief (which is at the heart of storytelling) to make visual, auditory, and social cues all work together with the semantic content of text or speech to make our communication richer, and to make the actors responsive and believable.

Which brings us to the importance of real-time inference to Charisma. If a player says (or does) something, the reaction needs to be immediate, otherwise we simply don't believe the conversation. The solution is a combination of compute power, smart caching and creative design, which are combined in Charisma through a no-code interface for creative game designers and producers to build these projects rapidly – the magic is hidden behind the scenes.

One of the more obvious pieces of magic, however, is the large natural language models that need to run in production environments. This needs MLOps knowhow, specific data sets for dialogue, and smart methods to bring down the cost of deploying models in the cloud.

"And this is where Pipeline.ai comes in. The quality of the model and the ability to run it at speeed with little latency is key. Mystic are not only leading in the field of delivering serverless compute for machine learning but they are deeply embeded in the LLM ecosystem - and that's important to us as things are moving very quickly!"

What’s next for Charisma?

Charisma is a creativity-first AI organisation. This means that they believe that AI should be a technology that inspires new forms of storytelling, entertainment, and education. It means that as new forms of generative AI come online, they have already been thinking about how this will affect Hollywood, the games industry, writing and beyond. It means that they partner with teams like the Entertainment Technology Center at the University of Southern California to explore how to design and script well-known characters with GPT-3 in ways that are not gimmicky and temporary, but that are engaging in stories that might last days, weeks, even years. As Guy says;

"These are the foundations we are building for the creative sector, because these are the foundations that are required for a stable AI sector in the future, and one that is truly inclusive.  Systems like Dalle-2, MidJourney and Stable Diffusion are like cameras at the birth of photography. Each has their own style, quirks and uniqueness."

Systems like Dalle-2, MidJourney and Stable Diffusion are like "cameras at the birth of photography". Each has their own style, quirks and uniqueness. The next step will naturally be from static image production to moving image production, and that's where Charisma will be powering the stories that underpin this new visual content.

See the Charisma Showcase and sign up to get early access to The Kraken Wakes

Keep up to date on Linkedin, on Instagram and on Twitter.

ABOUT PIPELINE.AI

Pipeline AI makes it easy to work with ML models and to deploy AI at scale. The self-serve platform provides a fast pay-as-you-go API to run pretrained or proprietory models in production. If you are looking to deploy a large product and would like to sign up as an Enterprise customer please get in touch.