A computerassisted constraintbased system for assembling fragmented objects

A computer-assisted constraint-based system for assembling fragmented objects G. Palmas, N. Pietroni, P. Cignoni, R. Scopigno Digital Heritage 2013 International Conference - 2013

Introduction Ancient artifacts are found fractured because of: • natural intervention (earthquakes, floods), • human intervention (war, neglect). Main task: assembling of fragmented objects. Process usually executed by: → archaeologists → restorers

Introduction Two different approaches: • Manual approach • executed directly on real fragments. • Computer-assisted: • CGI involved in the reassembly process. . • digital representation of original fragments. • free from physical constraints (typical of classical approaches).

The Classical Approach: Manual Reconstruction Process subdivided into: • on-site cataloguing of discovered fragments, according to precise characteristic features (ceramic pots: neck, base, decorations) • off-site examination of interesting fragments • reconstruction hypothesis • restoring gluing matching pieces A. R. Willis et al. Computational Reconstruction of Ancient Artifacts. IEEE Signal Processing Magazine, 25(4): 65 -83, July 2008.

Issues in classical approaches: A. R. Willis et al. Computational Reconstruction of Ancient Artifacts. IEEE Signal Processing Magazine, 25(4): 65 -83, July 2008. • • • time consuming: manual cataloguing physical risk: fragile fragments could be damaged during the process impossible reconstruction: if dispersed artworks no reversibility rehearsal very time-consuming

Virtual Reconstruction: Automatic methods Reconstruction process free from the physical constraints of the classical approach The 3 D models of the involved fragments are usually acquired using 3 D scanners: Reconstruction based on the automatic identification of fractured surfaces Matching of compatible faces using descriptors such as: • shape descriptors • color descriptors Qi-Xing Huang et al. Reassembling fractured objects by geometric matching. ACM Transactions on Graphics, 25(3): 569, July 2006.

Virtual Reconstruction: Automatic methods Difficulties of automatic methods: • Shape descriptors not significant for eroded pieces • Lack of matching candidates in case of missing pieces

Virtual Reconstruction: Semi-Assisted Methods Exploit user's experience in the reconstruction process • possible interactions: • modification of an automatically computed solution • specification of the position of each fragment N. Mellado et al. Semiautomatic geometry-driven reassembly of fractured archeological objects. VAST, 2010. • the user can identify correspondences between over-eroded fragments • wrong correspondences can be discarded faster than an automatic system

Virtual Reconstruction: Semi-Assisted Methods Difficulties of current semi-assisted methods: • only pairwise matching • potential errors of reconstruction are not verified Try to follow a human like approach: • hierarchical reconstruction

Our Method • A semi-assisted method: • the user species pairs of correspondences (constraints) between different fragments and also their importance (stiffness) • the system minimizes the distances between all pairs of samples belonging to constraints • possibility to work hierarchically: • matched fragments can be grouped to form a new rigid object • a group can be made of other groups

The Scenario • the sum of the distances between the extremities of the constraints identifies a global energy E • the system minimizes E and finds a rigid transformation for each involved entity in order to reassembly the object. • if the involved entity is a group, such transformation is applied to all of its children.

Energy Formulation Global energy: extend the formulation above to all the fragments. • All the contributes are inserted into a linear system. • The solution of such system produces a rigid transformation for each involved entity (rotation + translation).

Energy Formulation The properties of the rotation matrices can not be expressed linearly: • the provided transformation is not guaranteed to be rigid. • the rotation matrix closest to the computed one is found iteratively.

Solving Sequence: sequence of application of the computed transformations which terminates with the reconstruction of the original object • the system calculates and applies the transformations to the single fragments • the system executes the same procedure to the groups formed by the entities of the previous step • such process is then repeated until the complete object is reassembled

Solving Sequence Important: global optimization needed at the end of the process Split all the groups → simultaneous optimization of all the constraints The other semi-assisted methods minimize only pairwise

System Organization Involved entities: • 3 D models of fragments • groups of fragments or other groups • constraints System modeled as a graph: • nodes: fragments/groups • edges: constraints

Tree Structure • A Solving Sequence must be unique: • the graph must be a tree. • the fragments are the leaves. • the groups are the intermediate nodes. • the root is the complete object.

The Interface

The Interface designed to: • fully interact with the graph: • add/remove fragments/groups • add/remove/modify constraints • specify the extremities of new constraints between rendered nodes

The Interface Residual energy visualization:

The Interface The constraint of avoiding the interpenetration between pieces is not linear and is computationally hard → interpenetration detection using GPU An example of interpenetration visualization:

Case Studies Madonna of Pietranico: • terracotta statue of the 15 th century which was severely injured during the earthquake of 2009 • 16 pieces, 5 for the inferior part and 11 for the superior one

Case Studies The Fountain: • a table-fountain made of hardpaste porcelain of the 18 th century from the Victoria and Albert Museum of London • 4 pieces (so far)

Contribution Our method: • User driven, not rely on geometry matching • hierarchical approach (groups of matching fragments). • possibility to minimize potential reconstruction errors. • feedback of reconstruction errors. • an interface to guarantee a full interaction with the system.

Future Work • possibility to handle missing pieces • consider interpenetration as an active constraint to improve the provided solution • preview of pieces • extension with two further constraints: • Glue: constraint between two areas of two different fragments • Plaster: constraint of surface continuity between two different fragments: • improvements from suggestions of CH scholars.

Thanks for your attention! The development of this tool has been financed by the EG 7 FP IP “ 3 D-COFORM” project (2008 -2012, n. 231809)

Depth Peeling