First, we created a 1980s digital human we call 'Sophie'.
We were inspired by our own memories of messy bedrooms, walls covered in posters and album covers, concert tickets and stubs stuck in the mirror frame – and of course instruments. Sophie felt like an aspiring guitarist that enjoyed a wide range of music genres, from punk and heavy metal to pop and blues. But all of it was centered on Rock and Roll.
Sophie, and AI guitarist practicing in her messsy 1980s bedroom
Sophie, an AI musician studying some sheet music
Sophie, an AI musician walking down a grungy sidewalk near graffiti covered walls, heading to a gig.
Sophie, an AI guitarist getting warmed up on stage.
Sophie, an AI guitarist perfoming and having fun.
Sophie, an AI guitarist, connecting with her fans
But the question came up: Can she really play? Well, she's not that great yet, but she's getting better every day.
Sophie, an AI guitarist practices some simple riffs.
Sophie, an AI guitarist, practices some basic chords.
The 'tech' that is bringing Sophie to life
As usual, we started with a story and an artistic concept. Then we generated some stills using Flux1-dev in ComfyUI. We concentrated on the prompt engineering to get the flavor right. After dozens of generations with random seeds and a parameter tuning matrix, we selected the character that we liked best.
Then, working with our collaborator, professional photographer Ralph Palumbo, we created custom 'style' LoRAs that captured the lighting, composition, and scenes we wanted to explore. Next up was the creation and refinement of a character LoRA, for which we used Google Flow with VEO 3 to generate orbital video clips with a variety of expressions and viewing angles. Some custom scripting allowed us to extract still frames from the clips.
Our scripts connected to Ollama running Gemma3:27b Multi-Modal LLM to caption these new frames. Then we loaded our dataset into AI Toolkit (ai-toolkit) for Flux LoRA finetuning. A few hours later our RTX Pro 6000 Blackwell spit out some great samples and a solid checkpoint, and Sophie was ready for production.
But we didn't stop there. We realized that what we created was not just a digital character, but more of a digital actress. Stay tuned, as we're excited to be actively trying to cast her in some other roles.
If you are interested in deeper technical details and workflows, drop us an email. We're considering putting together some documentation and/or a video tutorial.
