Computational Intelligence Methods Decision Support Tools in Cultural

Computational Intelligence Methods & Decision Support Tools in Cultural Materials Anastasios Doulamis, Anastasia Kioussi, Maria Karoglou, Klio Lakiotaki, Ekaterini Delegou, Nikolaos Matsatsinis and Antonia Moropoulou National Technical University of Athens, School of Chemical Engineering

Computational Intelligence & Decision Support Tools • Assist experts to take solid decisions • Reject non-preferable solutions – Reduces the costs • Identify hidden knowledge – Image processing/analysis, computer vision • Improve validation performances • Results on optimal consolidation of cultural material National Technical University of Athens, School of Chemical Engineering

Outlines Supervised Learning Combined Fuzzy kmeans & Neural Unsupervised Learning National Technical University of Athens, School of Chemical Engineering

Optimal Consolidation of Cultural Heritage Material • Cultural heritage protection => targeted restoration actions to increase monuments’ lifetime. – conservation materials • The performance of each material on the restoration significantly differs with respect to its type, chemical properties and the building substrate. • Design phase: A decision support system which will assist the engineer to extract optimal conclusions. • Today, such section is expert-dependent process mainly exploiting her/his experience. National Technical University of Athens, School of Chemical Engineering

Computational Intelligence in Cultural Material Consolidation • We have applied different types of intelligent tools for optimal selecting the most suitable conservation materials Linear regression Supervised Neural Intelligence National Technical University of Athens, School of Chemical Engineering Fuzzy Kmeans & Neural Intelligence

AI & DSS for Conservation Interventions • Applications to two Cases of Conservation Interventions – Consolidation of Materials/Structures – Cleaning of architectural surfaces National Technical University of Athens, School of Chemical Engineering

Consolidation interventions How to support the decision making in choosing the most appropriate consolidation material The consolidation materials and interventions used intend to the : q Modification of micro structural characteristics of the stone, leading to lessening of stone susceptibility to salt decay q Prevention of decay due to grains de-cohesion q Amelioration of mechanical characteristics of the stone Main categories of consolidation products are: • Inorganic Materials • Nano-limes • Organic Materials • Alkoxysilanes National Technical University of Athens, School of Chemical Engineering

Validation in Lab and in the Monument Scale National Technical University of Athens, School of Chemical Engineering DDS OUTPUT

Parameter Climax Availability in Greece Parameter Penetration depth Unknown / Not satisfactory / depend on any factors/ depends on the solvent / good / very good with the use of specific solvent / very good/ Color change No change /depend on the surface / depend on the excess of material at surface/ medium / high/unknown Yes/No / Unknown Irreversibility 0 -10 Durability at environmental loads Chemical compatibility Climax 0 -10 Capillary absorption of water Low / medium / high/unknown 0 -10 Change of hardness Low / medium / high/unknown Standards Yes/No / Unknown Creation of film 0/1/2 National Technical University of Athens, School of Chemical Engineering

Criteria Adopted Availability Inversibility Resistance to Chemical Standardized Environmental Compatibility Rules Loads Yes/No Numerical /Unknown Values Filming Penetration Discolored Water depth Numerical Quality Values National Technical University of Athens, School of Chemical Engineering Binary Values Hardness absorption Quality Values

Experiments Set-Up • Two scenarios for different application substrate: • The ranking is primarily based on chemical composition of stones 1 st Scenario: Calcareous Stone (35 samples) 2 nd Scenario: Silicon-based Stone (34 samples) In future, more parameters will be included like micro-structural characteristics of material, mechanical properties etc, National Technical University of Athens, School of Chemical Engineering

A Feed-Forward Neural Network National Technical University of Athens, School of Chemical Engineering

Neural Networks Set-Up • Three categories (preferred, neutral non -preferred) • Two categories (preferred, nonpreferred) Continuous Outputs • Preference Order Quantized Outputs National Technical University of Athens, School of Chemical Engineering • Two layers networks • Constructive training method Different Network Sizes

Results- Generic Results in Training Set Results in Test Set Average Error over 10 randomly partitioned datasets when 20 hidden neurons are selected Training error Testing Error 1. 5% 23. 7% National Technical University of Athens, School of Chemical Engineering

Effect of Network Size-Generic Effect of Neural Size National Technical University of Athens, School of Chemical Engineering

Results- Scenario 1 test_output Average Error over 70 randomly Results in Test Set National Technical University of Athens, School of Chemical Engineering partitioned datasets when 20 hidden neurons are selected Training error Testing Error 1. 2% 22. 4%

Combined Fuzzy K-means & Neural Networks Testing on Data Neural Networks (supervised ) Fuzzy K_Means (unsupervised) National Technical University of Athens, School of Chemical Engineering

Results –Scenarios 1, 2 Average Error over 70 randomly partitioned datasets when 20 hidden neurons are selected SCENARIO 1 SCENARIO 2 Training error Testing Error 0. 4% 5. 7% National Technical University of Athens, School of Chemical Engineering Training error Testing Error 0. 7% 2. 6%

The UTA* algorithm AR Alternative Reference set Acronal 500 D Conservare® H 100 Ludox HS 30 Mowilith 30 Paraloid B 72 g 3 Reversibility Availability Hardness 6 yes 2 6 3 4 5 Ranking order 1 g 1 low Penetration depth good k=1 no low satisfactory k=2 7 2 2 no yes medium large 7 large good dependent Very good k=3 k=4 k=5 i=1 i=2 i=3 i=4 1 2 4 g 4 National Technical University of Athens, Post-optimality analysis School of Chemical Engineering 3 Criteria weights 19

Results-Scenario 1 Criteria Adopted Availability Inversibility Resistance to Chemical Standardize Compatibilit d Rules Environmen y tal Loads Yes/No Numerical Binary /Unknown Values Filming Penetration Discolored Water Hardness depth absorption Numerical Quality Values Values National Technical University of Athens, School of Chemical Engineering

Results-Scenario 2 0, 1207045438 0, 0968168892 953537 37404 Scenario 1 availability in Greece 0, 0978484873 reversibility 333175 0, 0543248145 337575 durability 0, 0974348195 545132 chemical compatibility 0, 0907898176 standards 222752 film 0, 0884421451 818158 0, 0483527490 965531 0, 2996857338 80365 penetration depth Scenario 2 0, 0085000000 884463 0, 0625147105 306808 0, 1207959540 91202 0, 1082930098 32017 0, 0677675224 322709 0, 1148439506 75972 color change water absorption National Technical University of Athens, School of Chemical Engineering availability in Greece reversibility durability chemical compatibility 0, 1037923212 standards 25367 film penetration depth color change 0, 0301425935 412286 water absorption hardness 0, 3236307963 0, 0597191412 08427 743917

Porous Biocalcarenite Pilot scale treatments for porous stone consolidation in the Medieval City of Rhodes Materials LUDOX HS 30 (PL) Silbond HT 20 (PH) Rhodorsil RC 70 (RP) Acryl Siliconic Resin (EU) National Technical University of Athens, School of Chemical Engineering

Evaluation of the Compatibility of Conservation Interventions in lab Changes of water absorption curves (capillary) of consolidated porous National Technical University of Athens, stones monitoring by infrared thermography in the laboratory School of and Chemical Engineering

IR Thermography Investigation of Capillary Rise, Monument Scale Investigated Surface: Gate of St. Paul, Medieval Fortifications of Rhodes Evaluation of Pilot Consolidation Interventions, Monument Scale Investigated Surface: Entrance of Moat, Medieval Fortifications of Rhodes 15 months after the applications 28 months after the applications Consolidation Materials: LUDOX HS 30 (PL), Silbond HT 20 (PH), Rhodorsil RC 70 (RP) acryl siliconic resin (EU) University of Athens, National Technical School of Chemical Engineering

Validation of the results -in laboratory (various analytical techniques like capillary absorption test, mercury intrusion porosimetry etc) -in monument scale (non-destructive testing) Feedback: Changes in materials ranking National Technical University of Athens, School of Chemical Engineering

Computational Intelligence on Cleaning Interventions Assessment • We have applied the aforementioned methodology for supporting the decision making on the assessment of pilot cleaning interventions on marbles surfaces • The application sites are located on the historic buildings of National Library of Greece (NLG), and National Archaeological Museum in Athens-Greece (NAM) • The diagnosed decay patterns are black grey crusts, washed out surfaces, and fractured surfaces of marble. National Technical University of Athens, School of Chemical Engineering

Presentation of Applications Sites Smooth Marble Architectural Surface NLG facade Different protection degree from rain wash North Facade National Technical University of Athens, School of Chemical Engineering

Presentation of Applications Sites Relief Marble Architectural Surface East Side, Full Protected from rain Wash NAM West Façade, North Column Relief Marble Architectural Surface Column Capital East Side, Full Protected from rain Wash Relief Marble Architectural Surface North Side, Different Protection Degree from Rain Wash National Technical University of Athens, North Column East Side School of Chemical Engineering

Decay Diagnosis - NLG Depending on the protection degree from the projected horizontal geison FOM Friable black -grey crust Cohesive black-grey crust Inter-granular fissured marble National Technical University of Athens, School of Chemical Engineering SEM FTIR

Decay Diagnosis - NAM FOM SEM Black-grey crust East orientation FTIR Washed out surfaces FOM SEM North orientation FTIR National Technical University of Athens, School of Chemical Engineering

Decay Diagnosis - NAM Black-grey crust of great variety FOM regarding: the width, the presence or not of FOM SEM Barite (patina), the location in the anthemia relief, relief side East orientation Relief face right part FTIR Relief side, central part Relief face central part FOM SEM FTIR National Technical University of Athens, School of Chemical Engineering FTIR SEM

In situ application of pilot cleaning interventions – Monument scale - NLG Smooth marble architectural surfaces, black-grey crust, inter-granular fissured marble Pab 22, P. ΑΒ 57, 2 h Pnc 22, P. (NH 4)2 CO 3, 2 h Pm 22, P. Mora, 2 h Ps 32 P. Sepiolite, 3. 5 h Pat 2, Atomized water Pm 24, P. Mora, 1. 5 h Ps 34 P. Sepiolite, 3 h Pab 24, Pnc 24, National P. Technical University of Athens, ΑΒ 57, 1, 5 h P. (NH 4)2 CO 3, 1. 5 h School of Chemical Engineering

In situ application of pilot cleaning interventions – Monument scale - NLG Smooth marble architectural surfaces, black-grey crust, inter-granular fissured marble Pnc 12, P. (NH 4)2 CO 3, 1 h Pab 12, P. ΑΒ 57, 1 h Pab 14, P. ΑΒ 57, 1 h National Technical University of Athens, Pnc 14, P. (NH 4)2 CO 3, 1 h School of Chemical Engineering Ped 12, Π. EDTA, 1 h Ped 14, P. EDTA, 1 h

In situ application of pilot cleaning interventions – Monument scale - NAM Relief marble architectural surfaces, black-grey crust, east orientation Ke Ke Ion exchange resin with solution of (NH 4)2 CO 3, 40 min Ke 2 a Ke 3 b Π. ΑΒ 57, 5 min Ion exchange resin with deionised water, 30 min Biological poultice Ke Ke 1 c Wet micro-blasting method • Spherical particles of Ca. CO 3 d<80μm, • Function pressure 0. 5 bar, • Proportion of Ca. CO 3/water: 1/3, • d nozzle 12 mm • working distance 50 cm University of Athens, National Technical School of Chemical Engineering Ke. G 3

In situ application of pilot cleaning interventions – Monument scale - NAM Relief marble architectural surfaces, washed out surfaces, north orientation Kn P. ΑΒ 57, 5 min Kn 3 c Ion exchange resin with deionized water, 40 min Kn 3 b Ion exchange resin with deionized water, 10 min Kn 2 a Ion exchange resin with solution of (NH 4)2 CO 3, 20 min Kn 1 a Ion exchange resin with solution of (NH 4)2 CO 3, 10 min Kn 1 b Double application of Ion exchange resin with solution of(NH 4)2 CO 3, 2 x 10 min Kn 1 c Kn 2 b Ion exchange resin with deionized water, 20 min Kn 2 c Double application of Ion exchange resin with deionized water, 2 x 20 min Kn 2 d National Technical University of Athens, School of Chemical Engineering Wet microblasting method

In situ application of pilot cleaning interventions – Monument scale - NAM Relief marble architectural surfaces of capital, east orientation Kke Ion exchange resin with solution of (NH 4)2 CO 3, 10 min Ion exchange resin with solution of (NH 4)2 CO 3, 40 min Ion exchange resin with deionized water, 60 min Kke 1 a Kke 2 a Kke 3 a 1 Kke 3 a 2 Ion exchange resin with deionized water, 10 min Ion exchange resin with deionized water, 20 min Wet micro-blasting method kkeg 51 Kke 1 b Ion exchange resin deionized Technicalwith University of Athens, water, 30 min National School of Chemical Engineering Kke 2 b Kke 3 b Ion exchange resin with solution of (NH 4)2 CO 3, 20 min kkee 3 ba P. ΑΒ 57, 5+ 15 min

Assessment of Cleaning Interventions: Techniques & Parameters Applying, after cleaning the same experimental techniques that were applied before cleaning, a methodological approach for cleaning assessment is compiled. Comparison of the marble surfaces physico-chemical characteristics before & after cleaning, along with recording the variations of the corresponding critical parameters, makes feasible the recommendation of the best cleaning according to the examined case. Digital Image Processing of SEM images: fracturing of the surface Shape factor (a roughness factor) SEM-EDS: chemical & mineralogical composition stratification, total crust width, patina, macro-crystalline gypsum layer width, micro-crystalline gypsum layer width Fracture Density Patina preservation index Friability index Preservation index of gypsum layer National Technical University of Athens, School of Chemical Engineering

Assessment of Cleaning Interventions: Techniques & Parameters Laser Profilometry: texture & roughness assessment Roughness Rq (μm) Surface area, (ratio of actual to projected area) Colorimetry CIELab color space: evaluation of color modifications L, Luminosity total colour difference ΔE difference in red-green blue-yellow a* Technical University of Athens, b* National School of Chemical Engineering

Assessment Criteria & Critical Assessment Parameters of Cleaning Interventions – Experimental Techniques Cleaning Assessment Criteria Chemical-mineralogical composition of the surfacesstratification Preservation of Patina, Preservation of Authentic Material, Scanning Electron Microscopy with Energy Dispersive X-ray Spectroscopy Color Texture, Morphology & Surface Cohesion - Surface Microstructure Removal of Black Depositions Surface Preservation State – Decay Susceptibility – Durability Roughness, Rq, Fracture Density Ratio of actual to projected area - Surface area Colorimetry Laser Profilometry Digital Image Analysis of Scanning Electron Microscopy Images Experimental Techniques for Measuring Critical Assessment Parameters of Cleaning National Technical University of Athens, Critical Assessment Parameters of Cleaning School of Chemical Engineering Total Color Difference, ΔΕ Colorimetry

Monitoring of Surface Preservation State – Durability of Marble 1. 2. Decay patterns distribution on the building are mainly controlled by material location, orientation, protection from rain-wash, atmospheric conditions and pollution. However, the long-term aesthetical and structural properties of marble are closely related to the lateral and vertical distribution of particulate matter and salts-gypsum, as well as to the bonding of the calcite grains in the matrix; factors that are strongly affected by cleaning. Therefore there is an urgent need for a tool to interrelate information – data, between space and physical-chemical characteristics of building materialsmarble, taking into account their variation over time. The suggested methodological tool is a GIS platform National Technical University of Athens, School of Chemical Engineering

Materials Mapping in GIS, Façade, National Archaeological Museum Working scale: building’s facade materials mapping performed using GIS based on the acquired data by NDT and inlab analytical techniques. Acropolis ØThe area extend of each investigated material was calculated by the means ofof Athens GIS Ø Historic plaster area was 248. 56 m 2, whereas new plaster area was 13. 63 m 2 National Technical University of Athens, School of Chemical Engineering

Materials Mapping in GIS, Façade detail, National Archaeological Museum Working scale: building’s facade ØDigital decay mapping performed using GIS based on the acquired data by NDT and in-lab analytical techniques. ØBrown color depicts areas of coating total detachment and intense fracturing total area on west façade: 25. 56 m 2 Acropolis ØBlue color represents the areas of coating loose interface to the substrate total area on westof Athens façade: 219. 72 m 2 National Technical University of Athens, School of Chemical Engineering

Façade detail - National Archaeological Museum • material type • applied cleaning method • application details • application area • cost Acropolis of Athens Working scale: building’s facade National Technical University of Athens, School of Chemical Engineering

GIS thematic maps for decay & pilot conservation interventions Recording & ascribing attributes to features Attribute db, (physical-chemical data, indexes of building material preservation state, before and after conservation) GIS db, (topological characteristics like area, perimeter, adjacency, etc) Relational Data Base National Technical University of Athens, School of Chemical Engineering

Spatial Classification of Decay. Different Physical-chemical characteristics & spatial properties Decay thematic map for the capital surface, along with RDBs for both front & side anthemia surfaces RDB

Spatial Classification of Conservation Interventions Pilot conservation interventions’ thematic map for the capital surface, along with the RDB of the front anthemia surface RDB

GIS analysis using Boolean and logical operations on decay thematic map for the capital surface Spatial entity in compliance with the combined expression central area of the anthemia relief Which is the entity that comply with: 1) roughness ≥ 7, 2) fracture density ≥ 35. 3 RDB

Suggested Information Management System ANALYSIS & OPERATIONAL TOOLS RELATIONAL DATABASE (ATTRIBUTES) GIS SPATIAL DATA Using the continuous process of GIS platform datasets concerning building pathology & conservation interventions are recorded, correlated, distributed & attributed to space in different working scales during different time periods Support on decision making for cleaning assessment using Computational Intelligence

Results-Crust Results in k-Means National Technical University of Athens, School of Chemical Engineering Combined Results K-means, Neural

Results-Washed-out Results in k-Means National Technical University of Athens, School of Chemical Engineering Combined Results K-means, Neural

Results-Fractured Results in k-Means National Technical University of Athens, School of Chemical Engineering Combined Results K-means, Neural

Results-Overall Average Error over 100 randomly partitioned datasets when 20 hidden neurons are selected CRUST FRACTURED Training error Testing Error 5. 3% 18. 5% 3. 0% 11. 2% Average Error over 100 randomly partitioned datasets when 20 hidden neurons are selected WASHED OUT Training error Testing Error 4. 2% 13. 7% National Technical University of Athens, School of Chemical Engineering

Interoperable Description of Cultural Material Content • Cultural material should be stored according to standardized formats • • Interoperability, Unified Accessibility, Portability, Exchangeability Metadata: Description of data • Extended Markup Language Schemes (XML’s) • MPEG-7 (visual description schemes) • MPEG-21 (resources and rights descriptions schemes) • Database schemas (My. SQL, Oracle) • Tools for efficient Knowledge Search and Content Mining • Guidelines for new research efforts at European level • Great research effort, data alignment schemes, knowledge representation tools National Technical University of Athens, School of Chemical Engineering

Conclusions-Guidelines • Artificial Intelligence can support automatic decisions for • Cultural material consolidation during the design phase • Finding the degree of importance for criteria used • Supervised learning => Feedforward Neural Networks • Unsupervised learning => Fuzzy k-means • Linear Regression (UTA *) => degree of importance • Results on Cleaning • Validation Performances • Issues on knowledge systems for cultural material National Technical University of Athens, School of Chemical Engineering

Thank for Your Attention! National Technical University of Athens, School of Chemical Engineering