3D modelling of art and cultural assets - opportunities and challenges of the method.

Photogrammetry briefly explained

Photogrammetry is a method from the field of computer vision. The technique makes it possible to reproduce art objects as 3D models. For this purpose, a series of digital 2D object photographs is used. The photographs are then further processed with the help of software. By photographing an object from a specific position (the camera's location) and from several angles, the position and shape are indirectly calculated, making geometrically exact and three-dimensional reproduction possible. The more image points of an object are available to the programme, the more precisely an object can be reproduced. The process is often supplemented with other techniques such as laser scanning or radar measurements to generate additional data. This allows images, reliefs, sculptures and even entire rooms and buildings to be scanned. So, what are the chances when art and cultural assets are reproduced as 3D digital copies or even as generative 3D models, and what challenges go hand in hand with this?

Opportunities through photogrammetry

Photogrammetry paves the way to new knowledge and enables a new engagement with the past of our cultural history. In the field of research, the procedure has the great advantage of being more efficient and cost-effective compared to other methods. This is because models, which until now were often produced in industrial manufacturing, were much more expensive and complicated to produce due to the small quantities and specific individual parts. In addition, expensive transport costs and complex logistics are eliminated. Once the objects have been captured, their data can be recalculated at any time and supplemented with further image material. This means that the digital models can be easily expanded when new discoveries are made. This also means that the preparation and planning time associated with a trip as well as travel costs are reduced, as the objects no longer necessarily have to be explored on site. The analysis is thus carried out comfortably from the computer without losing the spatial context.

Digital visualisation brings new insights, as extensive objects in particular can be better understood as a whole. In addition, due to its accuracy and the multi-visibility of the objects, the technology often makes surface features visible that would escape the human eye or would only be visible under a certain incidence of light. For example, graffiti carved into the walls. The contrast to the wall faded over the years due to dust or abrasion. However, the camera recognises the carvings and can visualise them. The method is also useful for objects that are difficult to access, as it can be applied well from a distance, especially when they are integrated into a spatial context. For two-dimensional objects such as paintings, orthophotos are generated, a reproduction of the surface of the painting with the highest geometric precision. In this way, the brushstroke is recorded topographically and plastically and can be analysed in detail. This is a great advantage when searching for damage or cracks in the picture. It can also be used to gain knowledge about technical production processes. With the help of photogrammetry, entire rooms, including their texture (colourfulness), can be reproduced as a digital 3D model. By subtracting the texture, it is even possible to look «under the texture». This exposes the topographical shape of the substrate, allowing damage to be detected without actually having to remove the surface. The underlying structures also reveal a lot about the construction and manufacturing history. But changes can also be measured when a borrowed object is returned. The object is simply photographed again and is then compared with the previous photographs. In this way, the affected object can be restored systematically and at an early stage.

Digital databases also make it easier to compare recorded objects with each other. Consequently, different hypotheses can be tested and discussed. But also, the software-based merging of fragments as a digital reconstruction becomes a reality. These possibilities thus allow a more complete overall view of the destroyed object, making the state of knowledge more complete. One example is Egyptian necropolises, the components of which are usually scattered around the world in various museums, but also the damaged frescoes in the tomb of Sethos I in the Valley of the Kings, which the Egyptology department of the University of Basel, among others, is studying. The tombs were massively damaged mainly as a result of «Egyptomania». The «research groups» of the time tried to copy the paintings or even cut out entire wall elements by means of so-called «rubbing-offs». Very often these elements shattered during this dubious endeavour and remained in pieces on the floor. Today, the broken individual elements are photographed and reassembled by means of digital comparison. In this way, they can be made visible again in their original state and provide further information about the work as a whole. The data can therefore be helpful for restoration or reconstruction. Before intervening on the actual original surface, it is possible to test different variations by means of the 3D scans. Today, restoration interventions are kept to a minimum and are only carried out if sufficient clarifications have been made. This is because it is always an intervention in which other information is lost or falsified.

This is not the only reason why documentation is essential. Generating and collecting data means that artworks can be digitally archived. In the event of destruction, the objects can still be viewed and reconstructed three-dimensionally in the state in which they were photographed. The exact dimensions, colours and shapes will be preserved, so that destroyed art and cultural objects can theoretically be physically reproduced. You can read more about destroyed art in the art24 blog «3D reconstruction of destroyed art, cultural goods and destroyed cultural heritage». Digital reconstructions can therefore be used to produce physical reproductions, replicas or partial additions quickly, easily and cheaply. These are realised by a 3D printer or CNC milling. The process also provides novel and creative approaches in the exhibition and mediation context. For example, the models can be made accessible online or shown as realised 3D copies in exhibitions; this also allows them to be touched, which would be unthinkable with an original.

The Egyptology Department of the University of Basel has been working for years with the Factum Foundation Madrid and the «Theban Necropolis Preservation Initiative» (TNPI), the Egyptian team trained and led by the Factum Foundation. The latter completed the recording of the entire tomb of Seti I. this year. Thus, 21 years have passed since Factum began its work in Egypt. This is quite an achievement. Through this collaboration, an exhibition was held at the Antikenmuseum Basel in 2017. Using 3D scans and other technical methods, the team reconstructed parts of the rock tomb of Sethos I. as exact facsimiles and included them in the exhibition «Scanning Sethos. The Rebirth of a Pharaonic Tomb» (2017). The remarkable Hall of Columns E as well as the «Hall of the Beauties» were reconstructed on a scale of 1:1. 

Fig. 1: Exhibition view of "Scanning Seti: The Rebirth of a Pharaonic Tomb" (29 October 2017 - 06 May 2018) at the Antikenmuseum, Basel. Facsimile of Column Room J, created by Factum Foundation, photo: © Oak Taylor-Smith for Factum Foundation.

On the one hand, elements were reproduced in their actual state, but on the other hand, reconstruction attempts were also made, allowing visitors to experience both possibilities of 3D modelling. This gave them an insight into the testimonies of the past without actually having to travel to Egypt. This benefits the original tombs in particular, as tourist flows are reduced and less damage is done. The complete tomb can now also be viewed as a virtual tour. In the future, the facsimiles will be on display next to those of Tutankhamun's tomb, which were set up as an open-air exhibition at the beginning of the Valley of the Kings in 2014.

Fig. 2: Detail of the west wall of Room I, tomb of Seti I. Shaded representation of surface data captured with the Lucida 3D scanner, photo: © Factum Foundation | Theban Necropolis Preservation Initiative for the Ministry of Antiquities, Egypt.

Fig. 3: Detail from the west wall of Room I, tomb of Seti I. Colour data captured using colour photography, photo: © Factum Foundation | Theban Necropolis Preservation Initiative for the Ministry of Antiquities, Egypt

Fig. 4: Detail from the west wall of Room I, tomb of Seti I. Surface and colour data, merged, photo: © Factum Foundation | Theban Necropolis Preservation Initiative for the Ministry of Antiquities, Egypt.

Another advantage is that very fragile or deformable objects can be protected better by photogrammetry. This is because the recording method itself is contactless and is accordingly gentle on particularly sensitive objects. Furthermore, instead of the originals, the prints can be used for research or shown and loaned out in museums and exhibitions. This also circumvents the risk of damage to the original resulting from transport. Photogrammetry is thus a proven method for high-precision art reproductions and replicas in arbitrarily high resolution and accurate colour reproduction and opens up new dimensions. But what are the difficulties that the method entails?

Challenges of photogrammetry

Photogrammetry also has its limitations, or rather its difficulties. For example, photographic images can distort the actual appearance and thus also the impression of an object. The colour of the object depends on the lighting, but also on the quality of the camera or scanner. This is why colour correction by means of image processing software is often necessary. One problem, as was mentioned above, is the effect of light. Certain materials often look different when viewed under certain lighting conditions. This implies that the ambient lighting must match or be made to match with some effort and good equipment. But this makes the light an artificially staged one. Photographing itself is not always easy either, especially with in situ objects. For example, an object with a reflective surface is difficult to photograph in bright sunshine, and the visual effect of an object also changes with constantly changing light conditions. Therefore, only one light setting is often not enough to accurately reproduce an object in its full effect. Ideally, the historical lighting context of an object must be known if an authentic impression is to be reproduced. This means that different lighting scenarios must be recorded, especially if the object was exposed to certain light gradients or situations. Despite certain difficulties, there are also advantages here. For even light that was not historically present can provide exciting insights and reveal new findings. In application programmes, the light can even be changed, allowing an object to be viewed under various lighting conditions or by means of a light simulation. For didactic transparency, therefore, the light information and its context would theoretically also have to be explained during viewing.

Another problem is that not all objects can be digitised without difficulty, as they have to fulfil certain requirements. The object to be digitised must have distinctive features so that similarities between the individual images of the series can be identified. Shapes that are too complex are unsuitable, as hard-to-reach places do not allow the artwork to be captured 100 per cent and are very time-consuming. Especially when the object to be photographed has regularly recurring patterns, overlapping forms or structures that are not very expressive, such as a white, uniform surface, the method reaches its limits. In addition to the shape, the colouring, certain materials and surface textures are also well or not so well suited. For example, matt objects are easier to photograph than translucent, shiny materials. Impossible to capture so far are transparent materials like glass, enamel or crystalline material. Fabric objects are also only suitable to a limited extent, as they are deformable and the result is thus constantly distorted. The result is therefore always dependent on the size, shape and optical properties of the object of interest.

The materiality in the production of real models by using rapid prototyping also poses a challenge if a copy is to be produced that is as accurate as possible. This is because the machines used so far only offer a certain selection of materials: for example, plastic, metal, plaster, wax, acrylic glass, synthetic resin or thermoplastics. Therefore, not all objects are suitable for creating a copy. In addition, colour printers are still relatively expensive.

On site photogrammetric work also becomes difficult due to the climatic conditions. The time window is often limited because of the heat and humidity in the tombs. Therefore, data often have to be analysed independently of the location, which can of course influence the interpretation during examination. For direct experience of objects is still important for research. In addition, any technology must be set up with great effort, especially in the case of the necropolises, as they are located on the edge of the desert.

Of course, the institutions also have to clarify questions about the type and security of long-term storage as well as the archiving of the generated data. This is also accompanied by the question of the ecological footprint, as data storage consumes a lot of energy. Another point of discussion in this context is the copyright of the image data and models, which must always be clarified and defined: Who has access to the data and how is it distributed? How is the data used? When are they freely accessible and when are they commercialised and how? The data of the tomb of Sethos I. are the property of the Ministry of Antiquities in Egypt. This benefits from commercial uses and only with their permission can the data be made freely available for conservation and monitoring purposes as well as for scientific studies. Another important aspect is thus the cooperation and exchange between the experts of the different disciplines, but also between the countries in which the investigations take place. This is the only way to achieve results that are as accurate and fair as possible.

Original or copy

Finally, the value of the objects must always be clarified: Is it worthwhile to produce a model of a certain object and what is the purpose of doing so? Can the copies adequately represent or even replace an original? What does the copy take away from us and what does it give us? Can we do justice to the original with a copy, also in view of the materiality and the creative process? What does it mean when we can travel back in time and see, or supposedly see, incomplete objects in all their glory again today, when it is a reconstruction? This discussion of original and copy will be explored further in a follow-up article. What seems clear is that virtual 3D models and their realised copies are a gentle method of conveying and researching art and cultural assets, even if there are still some technical difficulties. Despite legitimate questions about originals and copies, the exhibition of the facsimile of Thutmosis III. at the National Museum of Archaeology in Madrid, for example, showed that they seem to fascinate people. The exhibition in 2004 attracted record numbers of visitors. For insights into the process of making facsimiles, the website of the Factum Foundation is recommended. We also recommend the video of the Royal Armoury in Stockholm (Swe. «Livrustkammaren»), which gives an insight into the production of their 3D digital copies. The objects can be viewed on «Sketchfab», a 3D modelling platform where 3D content can be published and shared.

Thanks go to Giulia Fornaciari from Factum Foundation for her helpful information and thoughts, and to Factum Foundation for making their images available for the blog article.

Glossary:

Computer vision: An interdisciplinary science that is situated between computer science and engineering. Digital images or videos are processed and analysed using various methods, which are intended to generate knowledge and information.

Graffiti: The word is derived from the ancient Greek «gráphein» and means «to draw» or «to write». In Italian, «graffito» originally refers to a «hatching». Thus, it refers to paintings or inscriptions that were carved into walls or objects. The technique was already used in many ancient cultures, especially in ancient Egypt.

Orthophoto: Illustration of a surface using the photogrammetric method, which is true to scale and without distortion.

Valley of the Kings: The Valley of the Kings was a necropolis in Ancient Egypt, which is now located near Luxor. So far, 64 tombs and pits have been discovered, among them the tomb of Sethos I., Tutankhamun, but also of Ramses II. or Ramses XI. 

Egyptomania: The term describes a great enthusiasm for ancient Egyptian culture in European countries. The modern enthusiasm for Egypt was brought about by tomb excavations commissioned by Napoleon Bonaparte between 1798 and 1801, but also by the discovery of Tutankhamun's tomb in 1922 by the Englishman Howard Carter and an exhibition of Nefertiti's bust in 1924. Many European countries sent their research groups to Egypt, whic