Part II: Making believable historic characters for museum VR

Museums and heritage sites pride themselves on authenticity – be that in the objects on display in exhibitions, accurate facsimiles or replicas, or physical reconstructions. Most people assume that for a virtual reality or ‘metaverse’ museum experience, that all digital characters would be 3D modelled avatars.

However, creating believable 3D models which would live up to the accuracy expected by museum visitors takes an extraordinary amount of time and resource.

For this reason, we often utilise our 3D capturing expertise to save modelling time whilst ensuring highly detailed digital characters that are appropriate for the virtual historic environments we create. Our team is skilled in photogrammetry, volumetric video and stereoscopic capturing.

We discussed how we used LiDAR scanning and photogrammetry to capture 3D objects to populate our VR scenes in our previous blogpost. Yet to capture people we utilise stereoscopic video.

Stereoscopic video is based on using two cameras set up next to each other and at the same distance to the subject to emulate the left and right eye. This creates plasticity and depth when the separate videos are composed next to each other in post-production, resulting in a “3D effect”, and allowing us to accurately capture an actor’s movements.

We use this technique to film characters on greenscreen to be able to freely place captures of humans in augmented or virtual environments. This gives us a detailed and believable digital character which looks suited to our historic scene, but without the hassle of 3D modelling or the risk of an uncanny avatar.

For all projects we aim to work with actors from the local community; however, for the Tower Museum VR project we were adamant that the accents of each character were accurate to the region their real-life counterpart came from, as Irish accents and dialects differ vastly. This increased the complexity of casting but resulted in an added level of realism and depth to each digital character.

Sourcing period-appropriate props and costumes is vital for creating convincing characters for museum VR. Museum visitors are used to the authenticity of artefacts and costumes in exhibitions, so the characters in digital interpretation need to appear as realistic.

Whilst 3D modelling can replicate suitable clothing, the heft and texture of materials like wool, linen and cotton move differently which is time consuming and tricky to replicate digitally. This is one of the many reasons we choose to capture real actors in real clothing, rather than modelling them from scratch – the end result is just so much more believable.

For the TIDE project we were fortunate enough to work with our local costumers: Bristol Costume Services. With hundreds of outfits including shoes, hats and accessories, they were able to swiftly design costumes for each of our characters, taking into account their wealth, age, and status.

Having clothes and props specifically chosen for their character also helped our actors really embody their roles, despite the short shoot.

For the recent TIDE project, our team flew over to Derry to conduct the stereoscopic greenscreen shoot at Ulster University Magee Campus. This was a wonderful opportunity to work with film students and share some of our experience. We liaised with the studio coordinator and local director Keith O’Grady to prepare the space for the shoot and organise the cast.

Our expert photographer Misha spent the day shooting each actor separately, to make sure we had a full 3D capture of them which we could then comp into the virtual scene.

Zubr’s Creative Director and co-founder Jack directed the offstage actors to run lines with those being captured, and encouraged larger than life performances that would translate well into the museum VR experience. Including personalised gestures, correct sight lines and idle poses like yawns or shifting from foot to foot on the spot, added to the realism of each digital character.

Once we were back at the studio, our producer and developers cleverly positioned each character into our 3D modelled scenes. This is a tricky process, necessitating regular tests to check that the 3D illusion pops correctly and that the shadows within the scene are adding to the 3D impression. It’s these subtle details which increase how believable a digital character really is, and which makes this technique far better than simply 3D modelling someone.