Monitoring Control Systems CSH 6 Chapter 53 Monitoring

Monitoring & Control Systems CSH 6 Chapter 53 “Monitoring and Control Systems” Caleb S. Coggins and Diane E. Levine 1 Copyright © 2015 M. E. Kabay. All rights reserved.

Topics in CSH 6 Ch 53 ØIntroduction ØChange & Security Implications ØSystem Models ØTargets & Methods ØLog Management ØData Aggregation & Reduction ØNotifications & Reporting ØMonitoring & Control Challenges 2 Copyright © 2015 M. E. Kabay. All rights reserved.

Introduction (1) Ø M&C systems involve q. Prevention q. Detection q. Response Ø Topics q. Prevention, Detection & Response q. Controlling vs Monitoring q. Control Loop q. Defining Scope & System Requirements 3 Copyright © 2015 M. E. Kabay. All rights reserved.

Introduction (2) Ø Monitoring systems provides basis for q. Quality control q. Anomaly detection Ø Key elements q. System log files q. Data reduction programs Ø Additional resources q. Application program log files q. Statistical analysis tools and knowledge 4 Copyright © 2015 M. E. Kabay. All rights reserved.

Prevention, Detection & Response Ø Cost-effective solutions to mitigate risks q. IDS (intrusion detection systems, CSH 6 Ch 27) q. IPS (intrusion prevention systems, CSH 6 Ch 26, 27) q. UTM (unified threat management) q. Anti-malware systems (CSH 6 Ch 16, 17, 41) Ø Detection: identifying problem Ø Response q. Monitoring system: logging, alarms q. Control system: change parameters Ø Failure to detect & respond may have business & legal implications (lack of due diligence) 5 Copyright © 2015 M. E. Kabay. All rights reserved.

Purpose of Monitoring & Control Systems Ø Who is doing what when? Ø Can contribute to self-regulation q. Knowing that actions are monitored can reduce harmful behavior q. Increases self-awareness http: //tinyurl. com/2 o 9 fr Ø Provide information for controlling system q. Limiting access in response to observations q. Changing conditions in response to trends Ø Serve forensic investigations 6 Copyright © 2015 M. E. Kabay. All rights reserved.

Controlling vs Monitoring (1) Ø Monitoring q. Periodically checking aspects of operating environment q. Encourages constant awareness and vigilance q. Spot anomalies or trends q. Predict and prevent problems and attacks Ø Control q. In this context, refers to comparing observations with policies and standards q. May be referred to as audits or assessments 7 Copyright © 2015 M. E. Kabay. All rights reserved.

Controlling vs Monitoring (2) Ø Monitoring modalities q. Continuous Mode üReal-time üFirewalls, IDS, IPS, Anti-malware q. Batch mode üPeriodic analysis üAssessments and audits Ø Controls q. Cobi. T* (CSH 6 Ch 44, 49, 53, 54, 67) q. Based on well-defined policies *Control Objectives for Information and Related Technology 8 Copyright © 2015 M. E. Kabay. All rights reserved.

Control Loop Ø Humans usually remain in control loop q. Controller q. Target system q. Bidirectional communication path q. Transmitted data Ø Some systems require automated response q. E. g. , dangerous breaches (gas pipeline) cannot wait for human intervention q. But others should be open loop and require supervisory decisions (e. g. , patch management) 9 Copyright © 2015 M. E. Kabay. All rights reserved.

Defining Scope & System Requirements Ø Management must define q. Extent of application (scope) q. Capabilities required for success (requirements) Ø Technical requirements depend on specific systems Ø Often require q. Hardware q. Software q. Intellectual property rights q. Training q. Personnel 10 Copyright © 2015 M. E. Kabay. All rights reserved.

Change & Security Implications ØRegulations, Policies & Frameworks ØChange Management ØConfiguration Protection ØPerformance Considerations 11 Copyright © 2015 M. E. Kabay. All rights reserved.

Regulations, Policies & Frameworks Ø Compliance requirements may determine specific needs; e. g. , q. HIPAA (CSH 6 Ch 71) q. GLB (CSH 6 Ch 64) q. So. X (CSH 6 Ch 54, 64) Ø Frameworks support M&C; e. g. , q. Cobi. T: Control Objectives for Information & Related Technology GLB: Gramm-Leach-Bliley Act HIPAA: Health Insurance Portability & Accountability Act So. X: Sarbanes-Oxley Act 12 Copyright © 2015 M. E. Kabay. All rights reserved.

Change Management Ø Immediate awareness of changes in operational status valuable Ø Can identify tampering with production code & data Ø Or can lead to identification of malware, attacks Ø Records serve for diagnosis, analysis & prediction 13 Copyright © 2015 M. E. Kabay. All rights reserved.

Configuration Protection Ø Changes in (production) systems require careful attention to detail q. Checklists q. Approved equipment & specific parameters q. Approved software & specific patches Ø Monitoring / logging systems simplify task of spotting unauthorized or incorrect changes q. E. g. , installation of unauthorized WAP (wireless access point) may generate unusual traffic (and threaten confidentiality) 14 Copyright © 2015 M. E. Kabay. All rights reserved.

Performance Considerations Ø Addition of monitoring hardware, software may affect performance q. Some systems run on host being monitored – may use system resources Performance Evaluation Time üProcess-table related üCPU, RAM q. Others connect to network üMay affect throughput Ø Avoid implementing new systems without performance trials Ø Don’t install during full production period 15 Copyright © 2015 M. E. Kabay. All rights reserved.

Volume Considerations Ø Decide how often to close log files q. Disk space not much of an issue now üIn 1980, a 120 MB hard disk cost U$25, 000 § Approx U$100, 000 in 2013 value § U$833/MB üIn 2015, a 3 TB Western Digital external hard disk cost $90 § ~U$7. 6294 E-5/MB ($0. 000076294/MB) § ~40% drop per year compounded over 35 years) q. Main issue today is preventing data loss if system or logging process crashes q. Especially important to fight ransomware Image shows HP 7925 120 MB drive c. 1980 (1980 cost U$25, 000) 16 Copyright © 2015 M. E. Kabay. All rights reserved.

System Models ØInternal, 1: 1, 1: N, Distributed ØAutomation & HMI ØSnapshots vs Real Time ØMemory Dumps 17 Copyright © 2015 M. E. Kabay. All rights reserved.

Internal, 1: 1, 1: N, Distributed Ø Internal – monitor/control itself Ø 1: 1 – 1 system monitors another; e. g. , firewall, fault -tolerant parallel systems Ø 1: N – central M&C system for many systems; reduces costs, improves efficiency (more centralized logging, review, audit) Ø Distributed – sensors & controls dispersed; central logging collector; ideal for heterogeneous systems 18 Copyright © 2015 M. E. Kabay. All rights reserved.

Automation & HMI Ø 24 -7 -365 systems need automated M&C Ø High volumes make manual inspection/response to alerts impractical Ø Human-machine interface (HMI) allows operator to communicate with and control system Ø Typically intervene for highly unusual events or patterns Ø IPS can interact to defend against dispersed attacks (e. g. , worms, DDo. S) 19 Copyright © 2015 M. E. Kabay. All rights reserved.

Snapshots vs Real Time Ø One-point-in-time records useful for q. Auditing q. Problem diagnosis q. Incident response q. Forensic analysis Ø Real-time monitoring & control q. Continuous sensing & response q. E. g. , industrial processes & systems such as gas pipelines or manufacturing systems q. On Web sites, include IDS & IPS q. Real-time log analysis intelligent pattern recognition 20 Copyright © 2015 M. E. Kabay. All rights reserved.

Memory Dumps Ø Overview Ø Diagnostic Utilities Ø Output to Magnetic Media or Paper Ø Navigating the Dump Using Exploratory Utilities Ø Understanding System Tables Ø Security Considerations for Dump Data 21 Copyright © 2015 M. E. Kabay. All rights reserved.

Overview of Memory Dumps Ø Files containing entire contents of RAM Ø Useful for debugging and forensics Ø Two types q. Obtained through diagnostic utilities (debuggers) in real time q. Captured after system shutdown from copies made to other media 22 Copyright © 2015 M. E. Kabay. All rights reserved.

Memory Dumps Ø Copy contents of RAM (main memory) q Typically taken after system failure q Useful in forensic research/analysis Ø Methods q Diagnostic Utilities (debug) üRead RAM without filesystem restrictions üOften include facilities for interpreting / representing system tables q Output to magnetic media or paper üPrinting difficult with large amounts of RAM üGenerally no longer print to paper 23 Copyright © 2015 M. E. Kabay. All rights reserved.

Navigating the Dump Using Exploratory Utilities Ø RAM too large to explore “manually” q. I. e. , by inspecting everything q. Suppose we use 256 characters x 88 lines = 22, 528 bytes/page q. Then 1 MB would take ~46. 55 pp q. So 2 GB would take 95, 325 pp q. If inspection rate were 1 minute per page (FAST), would take 66 days to read the dump once Ø Use utilities to navigate through tables at will Ø Search for strings 24 Copyright © 2015 M. E. Kabay. All rights reserved.

Output to Magnetic Media or Paper Ø Early systems allowed printing contents of RAM to paper; e. g. , 2 MB filled stack a few inches thick Ø Today’s capacities cannot reasonably be printed in totality q. Even PC RAM of 2 GB on paper could be several feet high Ø More reasonable to write to disk, DVD Ø Analyze from those media Ø Especially valuable in forensic examination q. Non-volatile, non-writeable media preferred 25 http: //www. columbia. edu/acis/history/701 -tape. html Copyright © 2015 M. E. Kabay. All rights reserved.

Diagnostic Utilities Ø System-level DEBUG utilities give complete access to RAM Ø Thus allow total bypass of system security q. Extremely powerful = dangerous tools q. Can copy or alter any portion of memory q. Usually access system tables by name, make changes q. Stop processes, alter priorities etc. Ø Critically important to control access to these tools q. Separation of duties – approval, supervision 26 Copyright © 2015 M. E. Kabay. All rights reserved.

Memory Dumps Ø Security important for dumps q. Much sensitive information in clear üPasswords, keys üConfidential data from databases etc. üClassified data q. Therefore must safeguard physical and electronic access Ø Label clearly and unambiguously to prevent accidental usage Ø Store securely in physically-restricted facilities q. Vault, safe q. ID & signature required for acces 27 Copyright © 2015 M. E. Kabay. All rights reserved.

Security Considerations for Dump Data Ø Be aware that dumps can be major security vulnerability Ø Contain cleartext versions of vast amounts of confidential and encrypted data Ø Includes I/O buffers such as input from keyboards and files or output to displays and files Ø Can be disaster to release dump Ø Serious question about whether vendor should be permitted to see memory dump 28 Copyright © 2015 M. E. Kabay. All rights reserved.

System Tables Examples of Critically Important System Tables Ø Process control block (PCB) – pointers to all the running processes (“Task Manager” listing in Windows) Ø Process tables – all current details for every process Ø Data stacks – variables for each process & stack markers showing trail of execution Ø I/O Buffers – data in transit Ø Memory-management tables Ø Inter-process communication (IPC) tables q. Flags, semaphores, status fields 29 Copyright © 2015 M. E. Kabay. All rights reserved.

Understanding System Tables Ø Operating systems differ in detail Ø Basic concepts similar Ø Key tables include q. Process control table q. Process tables q. Data stacks q. Buffers q. Memory management tables q. IPC tables 30 Copyright © 2015 M. E. Kabay. All rights reserved.

Targets & Methods of Logging ØOverview ØProcess Flow & Job Scheduling ØNetwork Connectivity ØEnvironmental Concern ØSystem State ØSystem Components ØProcess Activities ØFile System ØAccess Controls 31 Copyright © 2015 M. E. Kabay. All rights reserved.

Overview of Targets & Methods Ø Choices depend on specific context Ø Consider mission-critical operations / systems; e. g. , q. Process flow q. Job scheduling q. Network connectivity q. Environmental measurement q. System states q. System components q. Process activities q. Configuration settings q. File system information q. Access control 32 Copyright © 2015 M. E. Kabay. All rights reserved.

Process Flow & Job Scheduling Ø Batch job scheduler tracks jobs Ø Ideally, use centralized job scheduler/logger Ø May have to connect to remote systems Ø If necessary, plan for incremental, gradual migration q. Allow for adaptation, learning q. Reduce stress on mission-critical production systems 33 Copyright © 2015 M. E. Kabay. All rights reserved.

Network Connectivity Ø Devices, protocols, media Ø Network operations center (NOC) monitors q. Status of links q. Status of key devices q. Bandwidth utilization Ø Zigbee standard q. IEEE 802. 15. 4 standard q. Local, ad hoc network connectivity usually applied to M&C Ø Need to plan for distributed systems to interconnect 34 Copyright © 2015 M. E. Kabay. All rights reserved.

Environmental Concerns Ø Physical factors q. HVAC: Temperature, humidity q. Electrical power: voltage, amplitude (spikes, brownouts), continuity q. Fire, smoke, water threats q. Perimeter breaches (breakins, intruders, vandalism) Ø Critical for business continuity (see CSH 6 Ch 58) q. Ideally monitoring & trend analysis provides early warning q. Allows preemptive action to stop problem or initiate emergency responses 35 Copyright © 2015 M. E. Kabay. All rights reserved.

System State Ø Critical variable on target system q. E. g. , M&C system for electrical power grid looks at electricity flow & individual components of network (generators, transformers, transmission lines) Ø Software agents run on target (host) system & report to monitoring hub Ø Host intrusion prevention systems (HIPS) monitor nodes in network q. Centralized reporting q. Attack correlation q. Useful data for postmortem analysis 36 Copyright © 2015 M. E. Kabay. All rights reserved.

System Components Ø Track usage of specific elements q. CPU q. RAM q. Storage Ø Operating systems may include resources Ø Specialized software available Ø Data support trend anomaly analysis 37 Copyright © 2015 M. E. Kabay. All rights reserved.

Process Activities 38 Ø Process in particular execution of specific piece of code on specific CPU by specific user at particular time q. Process = {code X CPU X user X time} Ø Every process should be q. Known q. Authorized Ø Antimalware products monitor for unauthorized processes Ø May also monitor processes for chargeback systems q. Organizational users pay for their share of resource investment & operational costs q. Plus: useful in anomaly detection Copyright © 2015 M. E. Kabay. All rights reserved.

File System Ø Who is doing what to which data when? Ø Helps in diagnosing system / application errors Ø Log files have different types of records corresponding to different type of file activities q. More later…. 39 Copyright © 2015 M. E. Kabay. All rights reserved.

Access Controls Ø Recording who asks for and receives (or doesn’t receive) access to resources q. Critically important for security management q. May identify malefactors before they can do damage Ø Also generally supports resource management q. Identify anomalies q. E. g. , “Nurse Betty” has been logged on to terminal for 72 hours…. 40 Copyright © 2015 M. E. Kabay. All rights reserved.

Log Management ØLog Generation ØTypes of Log File Records ØAutomation & Resource Allocation ØLog Record Security 41 Copyright © 2015 M. E. Kabay. All rights reserved.

Log Generation Ø Log files are records of events q. Basic building block for M&C systems q. Digital audit trail q. Often not enabled by default Ø Many different types typically available q. Must configure logging appropriately q. May ignore some events; e. g. , opening utility file of no sensitivity Ø Transaction logs q. Often store copies of original records q. Plus copies of change instructions or images of changed records (takes more space) Ø Must define policies for log retention 42 Copyright © 2015 M. E. Kabay. All rights reserved.

Types of Log-File Records Ø Log file = audit trail Ø Many types (not discussed in detail in this presentation – see 53. 5. 2. 1 -18) 43 System boot System shutdown Process initiation Process termination Session initiation Session termination Invalid logon attempt File open File close Invalid file access attempt File I/O System console activity Network activity Resource utilization Central processing unit Disk space Memory Copyright © 2015 M. E. Kabay. All rights reserved.

Data Aggregation and Reduction Ø Centralized Data Stores Ø Filtered Queries Ø Analyzing Log Records 44 Copyright © 2015 M. E. Kabay. All rights reserved.

Automation & Resource Allocation Ø Keeping logs defined, organized and available contributes to effective & efficient system management Ø Data retention requirements growing q. Include log files in policies Ø Weigh retention policies and centralization / consolidation policies q. Scalability important q. Estimate operational / financial costs of collecting, analyzing & storing logs from disparate systems in central repository 45 Copyright © 2015 M. E. Kabay. All rights reserved.

Log Record Security Ø Protect log records against unauthorized access Ø Methods q. Access control lists (ACLs) q. Checksums q. Encryption q. Digital signatures Ø Chain of custody important q. Track all transfers q. Use secure off-site repositories 46 Copyright © 2015 M. E. Kabay. All rights reserved.

Analyzing Log Files Ø Volume Considerations Ø Archiving Log Files Ø Platform-Specific Programs for Analysis Ø Exception Reports Ø Artificial Intelligence Ø Chargeback Systems 47 Copyright © 2015 M. E. Kabay. All rights reserved.

Archiving Log Files Ø Decide how long to keep log files Ø Usually legal requirements Ø Establish definite policies Ø Monitor and enforce Ø Safeguard archives (environmentally-sound and secure storage facilities) 48 Copyright © 2015 M. E. Kabay. All rights reserved.

Platform-Specific Programs for Analysis Ø Each operating system can have particular variations in log file structure Ø Look for log-file analysis tools specific for your environment Ø GOOGLE provides wealth of references with keywords “operating system log file analysis” q. AWStats – GNU GPL q. Argus – Sun Solaris, UNIX variants q. Sawmill – Web-related files 49 Copyright © 2015 M. E. Kabay. All rights reserved.

Exception Reports 50 Ø Often impossible to examine all records q. May be millions of events in single log file Ø Need to break out unusual events Ø Can set filters to scan for unusual conditions Ø Systems define baselines events (the norm) and spot unusual ones Ø Human beings often scan the exception reports Ø Sophisticated systems use AI to spot patterns and anomalies http: //www. thehousehistorians. co. uk/Imag es/Books. gif Copyright © 2015 M. E. Kabay. All rights reserved.

Artificial Intelligence Ø AI systems can be based on statistical quality control (SQC) Ø Spot multi-sigma deviations; e. g. , q No more than one user logon in a thousand has used an ID from the accounting department between the hours of midnight and 06: 00 ü So why is “Ralph” trying to logon at 03: 30? q What’s more, “Ralph” has not had to try his password more than twice in 1523 logons ü So why is “Ralph” trying his 18 th password at this time in the morning? Ø Can handle more sophisticated patterns 51 Copyright © 2015 M. E. Kabay. All rights reserved.

Chargeback Systems Ø Log files used to allocate costs to all possible resource utilization; e. g. , q$0. 00001 /disk I/O; q$0. 00002/process initiation; etc. Ø Users receive itemized bills (e. g. , monthly) showing resource utilization Ø Promotes optimization with help of users Ø Can alert user to unusual events or misuse: “Why is our bill 3 times higher this month? ? ” q. Because there’s a serious error in your code; or q. Because you’ve been hacked! 52 Copyright © 2015 M. E. Kabay. All rights reserved.

Protecting Log Files Against Alteration Ø Checksums Ø Digital Signatures Ø Encryption Ø Physically Sequestering Media 53 Copyright © 2015 M. E. Kabay. All rights reserved.

Checksums Ø Can generate hash total and append to each record Ø Any change that does not use the right algorithm to change the checksum will be identified Ø If checksum includes data from previous record, chaining makes changes very difficult for attacker Ø Attacker has to recreate entire chain of records starting at modified or deleted one 54 Copyright © 2015 M. E. Kabay. All rights reserved.

Digital Signatures Ø Can sign an entire file using public key cryptography (PKC) q. Create checksum q. Encrypt using a private key q. Check by decrypting using public key Ø Check validity by recomputing signature and comparing value against decrypted original signature Ø See next slide for reminder of how PKC works -----BEGIN PGP SIGNATURE----Version: PGP 8. 1 Comment: Digitally signed by M. E. (Mich) Kabay, Ph. D, CISSP-ISSMP 55 Copyright © 2015 M. E. Kabay. All rights reserved.

Encryption Ø Can also just encrypt the entire file Ø Then an attacker who lacks the appropriate key can do nothing with the file at all except delete it 56 Copyright © 2015 M. E. Kabay. All rights reserved.

Physically Sequestering Media Ø Same principles apply to log files as to any other form of valuable data Ø Can make backups Ø Store media in secure, safe storage facilities q. Access controls q. Environmentally stable q. Fire-resistant Ø E. g. , q. Iron Mountain q. Archive. America q. Many others…. 57 Copyright © 2015 M. E. Kabay. All rights reserved.

Notifications and Reporting ØAlerts ØTrend Analysis and Reporting 58 Copyright © 2015 M. E. Kabay. All rights reserved.

Alerts Ø Crying “wolf” not good – don’t overwhelm operators with stream of minor alerts Ø Judge operational value of information Ø Out-of-band monitoring can detect errors undetectable by monitored system itself Ø Alerts: email, pager, phone, SMS Ø Human Machine Interface (HMI) q. Situational awareness q. Virtual buttons, meters, graphs q. Management dashboard to report on ignored alerts 59 Copyright © 2015 M. E. Kabay. All rights reserved.

Trend Analysis and Reporting Ø Analyze pace of security improvements Ø Consistency of internal controls Ø Peaks in violation of security policies between audits – danger sign Ø Chargeback (discussed earlier) can spark serious examination of trends Ø Exception reports identify anomalies 60 Copyright © 2015 M. E. Kabay. All rights reserved.

Monitoring and Control Challenges ØIndustrial Control Systems ØMobile Computing ØVirtualization 61 Copyright © 2015 M. E. Kabay. All rights reserved.

Industrial Control Systems (ICS) Ø Distributed Control Systems (DCS) q. Relatively autonomous, little human interaction q. E. g. , oil refineries Ø Supervisory control and data acquisition (SCADA) q. Extensive HMI q. Direct communication with programmable logic controllers (PLCs) q. Increasing use of networking q. Often unsecured logically and physically 62 Copyright © 2015 M. E. Kabay. All rights reserved.

Mobile Computing Ø Data in transit q. To/from PCs, laptops, tablets, phones & radiofrequency identification (RFID) systems q. Often over unsecured channels q. Must move to virtual private networks (VPNs) Ø Data at rest q. In PCs, laptops, tablet and phones q. Often unsecured q. Must move to data encryption Ø BYOD: Bring Your Own Device q. Increasing complexity for sysadmins q. Wide range of hardware & software to monitor & control 63 Copyright © 2015 M. E. Kabay. All rights reserved.

Virtualization Ø Virtualization supports hardware sharing q. Physical hardware q. Virtualization interface (VI) q. Virtual machines (VMs) üEntire operating systems or üSpecific applications Ø Hypervisors can support different VMs Ø Migration q. VMs can move from hardware device to device q. Must define and monitor security policies üE. g. , could prohibit hypervisor from managing internal, high-security systems & public, low-security systems on same hosts 64 Copyright © 2015 M. E. Kabay. All rights reserved.

Review Questions (1) 1. How can monitoring system data contribute to information assurance? 2. Which type of log file record includes information about the following events and how can you use these records for IA purposes? a) When the system started? b) When the system stopped? c) Who launched a process and when? d) Total amount of various system resources (CPU, I/O, swaps of VM, maximum priority, etc. ) used by a process during its lifetime? e) Who started a session on the system and when? f) Total system activity carried out by a user during a session? g) Number of bad passwords entered during logon attempts? h) Who opened which file at what time for which purposes? i) How much I/O a specific file was involved in while it was open? j) Who tried to access files in unauthorized ways? k) Detailed records of exactly what information was written into a database? l) What messages were sent to the system operator? m) Data about Internet connections? 65 Copyright © 2015 M. E. Kabay. All rights reserved.

Review Questions (2) 3. Why do most sites no longer worry about the disk space consumed by log files? 4. Whom should you consult when deciding on how long to keep log files? Why? 5. What are exception reports and why do we need them? 6. How can chargeback systems help us improve IA? 7. What mechanisms are there to protect log files against tampering? 8. Why are memory dumps highly sensitive from an IA perspective? 9. Why do we need special diagnostic utilities to navigate through today’s memory dumps? 66 Copyright © 2015 M. E. Kabay. All rights reserved.

Now go and study 67 Copyright © 2015 M. E. Kabay. All rights reserved.