The Zamani Project: Creating virtual heritage sites with Twinmotion and RealityCapture
Heritage provides a window into the past, helps us understand our present, and encourages us to plan for our future. Awareness of heritage helps to develop cultural identity, and promote tolerance and acceptance of others. However, heritage sites around the globe often have limited documentation resources, or none at all. These sites also face other threats: natural disasters, rising sea levels, vandalism, cultural terrorism, and mass tourism.
In recognition of the dire consequences of losing these influential and history-rich areas, in 2004 Professor Heinz Rüther of the University of Cape Town conceptualized the Zamani Project, a non-profit initiative that enables worldwide access to high-quality, accurate digital doubles of heritage sites, with a focus on Africa. The initiative also plays a key role in educating the public about the importance of heritage conservation.
Over the past 17 years, in collaboration with significant heritage organizations such as UNESCO and the World Monuments Fund, the Zamani Project team—led by Rüther as Principal Investigator—has digitally documented over 65 sites in 18 countries. Today, that digital documentation is taking the form of beautiful and engaging 3D animations and immersive experiences, with the help of Twinmotion and RealityCapture.
Rüther’s team includes Chief Scientific Officers Roshan Bhurtha and Ralph Schroeder, who have been with the project since year one, and Bruce McDonald, who joined five years ago.
Using Z+F laser scanners, together with DSLR cameras and drones, the team collects massive amounts of point cloud and photographic data from the site, which RealityCapture is able to process and merge into a single, very high-resolution 3D model.
In recognition of the dire consequences of losing these influential and history-rich areas, in 2004 Professor Heinz Rüther of the University of Cape Town conceptualized the Zamani Project, a non-profit initiative that enables worldwide access to high-quality, accurate digital doubles of heritage sites, with a focus on Africa. The initiative also plays a key role in educating the public about the importance of heritage conservation.
Over the past 17 years, in collaboration with significant heritage organizations such as UNESCO and the World Monuments Fund, the Zamani Project team—led by Rüther as Principal Investigator—has digitally documented over 65 sites in 18 countries. Today, that digital documentation is taking the form of beautiful and engaging 3D animations and immersive experiences, with the help of Twinmotion and RealityCapture.
Rüther’s team includes Chief Scientific Officers Roshan Bhurtha and Ralph Schroeder, who have been with the project since year one, and Bruce McDonald, who joined five years ago.
Recreating heritage sites with RealityCapture
To capture the heritage sites and digitally reconstruct them, the team uses RealityCapture—powerful photogrammetry software that enables you to create ultra-realistic 3D models from sets of images and/or laser scans. The product is now part of the Unreal Engine ecosystem that includes Twinmotion, after its makers Capturing Reality joined the Epic Games family in March 2021.Using Z+F laser scanners, together with DSLR cameras and drones, the team collects massive amounts of point cloud and photographic data from the site, which RealityCapture is able to process and merge into a single, very high-resolution 3D model.
The model is then simplified to reduce the triangle count to a manageable level, before normal maps and textures are extracted from the original data by RealityCapture and applied to the mesh. The result is a high-quality textured model ready for import to Twinmotion. The Unreal Engine export preset is used in RealityCapture to get the optimal results, producing an OBJ mesh at the correct scale for use in Twinmotion.
“The quality of the 3D models produced by RealtyCapture is excellent,” says McDonald. “They look amazing once imported into Twinmotion.”
“Twinmotion enables us to bring the heritage site to life for those who are unable to visit it,” says McDonald. “We strive to create animations and experiences that don’t just show a 3D model, but show the model in an environment that matches reality as closely as possible.”
“The quality of the 3D models produced by RealtyCapture is excellent,” says McDonald. “They look amazing once imported into Twinmotion.”
From 3D model to photoreal immersive experience with Twinmotion
Once the 3D model is in Twinmotion—the software has no problem importing the high-resolution asset complete with textures—the team can add the finishing touches to the environment, enabling the Zamani Project to enter the realm of storytelling.“Twinmotion enables us to bring the heritage site to life for those who are unable to visit it,” says McDonald. “We strive to create animations and experiences that don’t just show a 3D model, but show the model in an environment that matches reality as closely as possible.”
Zamani Project - Univerty of Cape Town
Twinmotion helps achieve this elevated level of realism through many of its features for photorealistic visualization. For example, the included library of Quixel Megascans provides access to high-quality scanned assets and materials to build true-to-life background environments, and you can import heightmap data to create a contextual landscape, both features McDonald values.
Moreover, it’s easy to ensure the sun’s position is correct at sunrise and sunset based on the real-life geographical location of the site. Taking the time to finesse details as small as these is crucial to the plausibility of the end result.
With the scene set, creating animations is easy in Twinmotion, says McDonald, citing the quality of the default lighting, the ease of controlling the camera to create video clips, and the ability to do things like have doors automatically open when a camera approaches.
Moreover, it’s easy to ensure the sun’s position is correct at sunrise and sunset based on the real-life geographical location of the site. Taking the time to finesse details as small as these is crucial to the plausibility of the end result.
With the scene set, creating animations is easy in Twinmotion, says McDonald, citing the quality of the default lighting, the ease of controlling the camera to create video clips, and the ability to do things like have doors automatically open when a camera approaches.
Twinmotion also enables the team to immerse themselves within the scene while editing it using a virtual reality (VR) headset, providing a level of realness previously out of reach.
Once the scene is complete, the team can export high-resolution still images, standard or 360° videos, panoramas and linked Panorama Sets, and even complete interactive 3D presentations, all from the same project. Using Twinmotion Cloud, Panorama Sets and Presentations can be shared with anyone anywhere in the world via a simple URL, which they can access from a web browser on a smartphone, tablet, or computer, without the requirement for high-spec hardware or the need to download anything. Local Presentations can also be viewed in VR.
Despite the power of its features, Twinmotion is renowned for its ease of use and shallow learning curve, which translate into greater productivity and faster results. “We are able to generate beautiful, high-quality animations very quickly in Twinmotion,” says McDonald. “Previously it took a long time in other software.”
Bunce Island is a 1,600-feet uninhabited island, established as a slave trading station in 1670. The fort went into disuse in 1808, and in 1840, the island was finally abandoned. In 1948, Bunce Island was declared a National Monument.
Once the scene is complete, the team can export high-resolution still images, standard or 360° videos, panoramas and linked Panorama Sets, and even complete interactive 3D presentations, all from the same project. Using Twinmotion Cloud, Panorama Sets and Presentations can be shared with anyone anywhere in the world via a simple URL, which they can access from a web browser on a smartphone, tablet, or computer, without the requirement for high-spec hardware or the need to download anything. Local Presentations can also be viewed in VR.
Despite the power of its features, Twinmotion is renowned for its ease of use and shallow learning curve, which translate into greater productivity and faster results. “We are able to generate beautiful, high-quality animations very quickly in Twinmotion,” says McDonald. “Previously it took a long time in other software.”
Bunce Island Fort, Sierra Leone
In 2021, the Zamani Project visited Sierra Leone for a World Monuments Fund project to document the Old Fourah Bay College in Freetown. After this, the team traveled up the Sierra Leone River to capture and recreate the Slave Fort on Bunce Island.Bunce Island is a 1,600-feet uninhabited island, established as a slave trading station in 1670. The fort went into disuse in 1808, and in 1840, the island was finally abandoned. In 1948, Bunce Island was declared a National Monument.
Last year, the Zamani Project team documented nearly the entire site within a day and a half, taking approximately 4,500 photos using a Nikon D7200 DSLR camera and more than 70 color laser scans from a Z+F 5010X scanner, which were then transformed into a plethora of materials—from 3D models and panoramas to still images, animations, and more—with the help of RealityCapture and Twinmotion.
Mapungubwe, South Africa
Another site the Zamani Project worked on was Mapungubwe. Located on the northern border of South Africa, neighboring Zimbabwe and Botswana, and at the joining of the Limpopo and Shashe rivers, Mapungubwe was considered to be the largest kingdom in the sub-continent before being abandoned during the 14th century.Remaining largely untouched, Mapungubwe encompasses palace sites and entire settlement areas from centuries past. It also boasts two areas recognized as earlier capital cities in a similar untouched state. This rare find enables insight into a society developing over 400 years.
Capturing Mapungubwe was part of a larger project to document heritage sites in South Africa, run by the South African Heritage Resource Agency (SAHRA). The ultimate aim was to create an animation to show at South Africa’s stall at Expo 2020 Dubai.
The Zamani Project team—accompanied by an armed park ranger at all times to ensure their safety against the lions, elephants, and other wild animals in the park—used a Nikon D7200 DSLR camera and Phantom 3 Pro and 4 Pro drones to photograph the Mapungubwe area, capturing around 10,000 photos in total. These were then uploaded to RealityCapture, merged into one coordinate system to create textured 3D models, and, finally, brought into Twinmotion for the finishing details. This process included dressing the environment with Baobab trees duplicated from a single scan, creating simulated water for a cistern, and adding stunning panoramic backdrops from the drone footage.
The Zamani Project team—accompanied by an armed park ranger at all times to ensure their safety against the lions, elephants, and other wild animals in the park—used a Nikon D7200 DSLR camera and Phantom 3 Pro and 4 Pro drones to photograph the Mapungubwe area, capturing around 10,000 photos in total. These were then uploaded to RealityCapture, merged into one coordinate system to create textured 3D models, and, finally, brought into Twinmotion for the finishing details. This process included dressing the environment with Baobab trees duplicated from a single scan, creating simulated water for a cistern, and adding stunning panoramic backdrops from the drone footage.
Next stop…Petra!
The Zamani Project team has firmly embedded RealityCapture and Twinmotion into its workflow and sees them as crucial elements of its future projects, alongside Unreal Engine, which it will bring into play on its latest venture. Taking place in Petra, Jordan, this most recent project has been made possible thanks to an Epic MegaGrant.The vast scale of the Petra landscape has meant that documenting it has already taken over a decade, with the team making initial surveys between 2011 and 2014. Armed with the MegaGrant funding and the latest capture technology, they visited Petra again in March 2022, and returned home with terabytes of data, including around 100,000 photos.
All of this data is now being processed in RealityCapture, before a 3D experience is created in Unreal Engine 5, where the team will make use of the engine’s Nanite virtualized geometry system to handle the huge meshes in real time. We can’t wait to see the results!