Avoiding uncanny characters to create realistic and authentic interactions in museum VR
Welcome to the second stage of creating realistic digital interpretation for museums. Once you’ve established the 3D environment your historic or museum VR experience is based in, you need to populate it.
Unfortunately this is where many projects leave the realm of authenticity and enter the uncanny…
Read on to discover how we ensured the interactive digital characters in our recent Tower Museum VR project with partners USI, TIDE Atlantic, and Derry and Strabane District Council were believable, inviting, and appropriate for the historic environment we created.
1: Character Creation
When it comes to developing a realistic museum VR experience, you need a vivid story filled with colourful characters to populate your virtual historic world. Without fully fleshed out backstories, motivations and pasts, it’s hard to make digital characters in VR seem authentic or multi-dimensional – pardon the pun!
For our Tower Museum VR project, we wanted to introduce users to a range of characters based on real people from journals and diary entries to show the breadth of experiences had by passengers emigrating from Derry to New York.
This allowed us to represent passengers with different incentives for travelling, who came from different cities, had different economic statuses and held different religious beliefs. All of these factors affected how the character was presented in VR.
2: Capturing techniques
Museums and heritage sites pride themselves on authenticity – be that in the objects on display in exhibitions, accurate facsimiles or replicas, or physical reconstructions. Most people assume that for a virtual reality or ‘metaverse’ museum experience, that all digital characters would be 3D modelled avatars.
However, creating believable 3D models which would live up to the accuracy expected by museum visitors takes an extraordinary amount of time and resource.
For this reason, we often utilise our 3D capturing expertise to save modelling time whilst ensuring highly detailed digital characters that are appropriate for the virtual historic environments we create. Our team is skilled in photogrammetry, volumetric video and stereoscopic capturing.
We discussed how we used LiDAR scanning and photogrammetry to capture 3D objects to populate our VR scenes in our previous blogpost. Yet to capture people we utilise stereoscopic video.
Stereoscopic video is based on using two cameras set up next to each other and at the same distance to the subject to emulate the left and right eye. This creates plasticity and depth when the separate videos are composed next to each other in post-production, resulting in a “3D effect”, and allowing us to accurately capture an actor’s movements.
We use this technique to film characters on greenscreen to be able to freely place captures of humans in augmented or virtual environments. This gives us a detailed and believable digital character which looks suited to our historic scene, but without the hassle of 3D modelling or the risk of an uncanny avatar.
3: Casting, Costumes and Props
For all projects we aim to work with actors from the local community; however, for the Tower Museum VR project we were adamant that the accents of each character were accurate to the region their real-life counterpart came from, as Irish accents and dialects differ vastly. This increased the complexity of casting but resulted in an added level of realism and depth to each digital character.
Sourcing period-appropriate props and costumes is vital for creating convincing characters for museum VR. Museum visitors are used to the authenticity of artefacts and costumes in exhibitions, so the characters in digital interpretation need to appear as realistic.
Whilst 3D modelling can replicate suitable clothing, the heft and texture of materials like wool, linen and cotton move differently which is time consuming and tricky to replicate digitally. This is one of the many reasons we choose to capture real actors in real clothing, rather than modelling them from scratch – the end result is just so much more believable.
For the TIDE project we were fortunate enough to work with our local costumers: Bristol Costume Services. With hundreds of outfits including shoes, hats and accessories, they were able to swiftly design costumes for each of our characters, taking into account their wealth, age, and status.
Having clothes and props specifically chosen for their character also helped our actors really embody their roles, despite the short shoot.
4: Shooting and Editing
For the recent TIDE project, our team flew over to Derry to conduct the stereoscopic greenscreen shoot at Ulster University Magee Campus. This was a wonderful opportunity to work with film students and share some of our experience. We liaised with the studio coordinator and local director Keith O’Grady to prepare the space for the shoot and organise the cast.
Our expert photographer Misha spent the day shooting each actor separately, to make sure we had a full 3D capture of them which we could then comp into the virtual scene.
Zubr’s Creative Director and co-founder Jack directed the offstage actors to run lines with those being captured, and encouraged larger than life performances that would translate well into the museum VR experience. Including personalised gestures, correct sight lines and idle poses like yawns or shifting from foot to foot on the spot, added to the realism of each digital character.
Once we were back at the studio, our producer and developers cleverly positioned each character into our 3D modelled scenes. This is a tricky process, necessitating regular tests to check that the 3D illusion pops correctly and that the shadows within the scene are adding to the 3D impression. It’s these subtle details which increase how believable a digital character really is, and which makes this technique far better than simply 3D modelling someone.
Bringing together all of these elements, we are able to create museum VR experiences with digital characters that appear authentic to the historic environments, but who feel approachable and believable. Instead of appearing ominous or frightening, they are welcoming and make VR visitors want to approach them.
Equally, these techniques result in detailed characters in a much shorter timescale and reduced budget compared to modelling from scratch. By opting for this cheaper and quicker approach we are able to make digital interpretation which truly serves the museum and heritage sector, without compromising on quality.
If you’re intrigued about working with us or have an idea for a project, why not check out our other digital interpretation for museums and historic sites, and feel free to get in touch with any questions.