COMP 1321 Digital Infrastructures Richard Henson November 2018

  • Slides: 40
Download presentation
COMP 1321 Digital Infrastructures Richard Henson November 2018

COMP 1321 Digital Infrastructures Richard Henson November 2018

Week 9: File Systems, Data Backup, Fault Tolerance n Objectives – Explain differences between

Week 9: File Systems, Data Backup, Fault Tolerance n Objectives – Explain differences between FAT 32 and NTFS file systems – Effectively use the features in Windows that aid data backup and rapid data retrieval

“If it can get lost, it will!” ………… anon

“If it can get lost, it will!” ………… anon

Hard Disk Matters n Also known as a volume… – can have a number

Hard Disk Matters n Also known as a volume… – can have a number of partitions – partitions can carry different file systems – “first” partition (normally C: ) can be “bootable” » can be used to load an operating system on that same partition n For addressing, volume divided into cylinders and sectors

File Systems, Sectors, Cylinders n Each type of file system uses that hard disk

File Systems, Sectors, Cylinders n Each type of file system uses that hard disk in a different way – each cylinder can carry a fixed number of bits n Bytes/sector depends on file system: – FAT 512 bytes per sector – NTFS: 4096 bytes per sector Very many cylinders In a single volume

Reminder of “Partitions” n The basic logical unit for storing data – applies to

Reminder of “Partitions” n The basic logical unit for storing data – applies to all storage devices – hard disks can have many partitions – most storage devices limited to one n A partition can only accept data once it has been formatted – formatting also determines the file system use to organise data e. g. FAT, FAT 32, NTFS

Any data that is stored by computer must be retrievable (!) n Software for

Any data that is stored by computer must be retrievable (!) n Software for managing data onto storage… “file system” – e. g. FAT 32, NTFS n Provides mechanism to – – index locations on the storage device put data into files mark locations where files are stored locate stored data so files can easily be retrieved into memory

Putting data onto a Partition 1. 2. 3. Data held in memory as a

Putting data onto a Partition 1. 2. 3. Data held in memory as a file taking up x memory locations Calculation made regarding where to fit the file on secondary storage partition Data sent from memory to storage memory CPU Secondary storage

Extracting data from a Partition 1. 2. 3. Data held in storage as a

Extracting data from a Partition 1. 2. 3. Data held in storage as a file taking up x locations Calculation made regarding where to fit the file in memory Data sent from storage into memory CPU Secondary storage

Files between storage devices File system software makes file easy to locate, via catalogue/index

Files between storage devices File system software makes file easy to locate, via catalogue/index n Retrieval (to memory as a stream of bytes…) n Saved to another storage device (B) n memory Device A (NTFS) CPU Device B (FAT 32)

Cloning a Disk Need to bypass the file system… n Every sector copied in

Cloning a Disk Need to bypass the file system… n Every sector copied in turn to memory… n – then copied back to device B n Lot of sectors… can take time! memory Device B Device A CPU

Basic Principles for Collecting Evidence Association of Chief Police Officers (ACPO) Guidelines on Computer

Basic Principles for Collecting Evidence Association of Chief Police Officers (ACPO) Guidelines on Computer Evidence. basic principles of acquiring evidence from computer systems. accepted by the courts in the United Kingdom.

ACPO Principle 1 No action taken by the Police or their agents should change

ACPO Principle 1 No action taken by the Police or their agents should change the data held on a computer or other media. Where possible computer data must be ‘copied’ and the copy examined.

ACPO Principle 2 • In exceptional circumstances it maybe necessary to access the original

ACPO Principle 2 • In exceptional circumstances it maybe necessary to access the original data held on a target computer. • However it is imperative that the person doing so is competent and can account for their actions.

ACPO Principles 3 An audit trail must exist to show all the processes undertaken

ACPO Principles 3 An audit trail must exist to show all the processes undertaken when examining computer data Many forensic tools record logs of processes performed and results obtained

ACPO Principle 4 The onus rests with the person in charge of the case

ACPO Principle 4 The onus rests with the person in charge of the case to show that a computer has been correctly examined in accordance with the law and accepted practice

Forensic Imaging Process Make a bit wise image of the contents of digital media

Forensic Imaging Process Make a bit wise image of the contents of digital media Store the original media and carry out forensic analysis using the copy image If necessary to switch on suspect machine; Restore image to another drive and install it in suspects machine Or mount and start in a Virtual Machine Retrieve evidence in a readable form

Image Hard Disk

Image Hard Disk

Forensic Examination Process Decide on best forensic tool(s) for the job Expand ALL compound

Forensic Examination Process Decide on best forensic tool(s) for the job Expand ALL compound files Hash ALL File Streams Perform File Signature Analysis Perform Entropy Test Generate Index and/or Thumbnails of Graphics Carve Data Carve Meta Data

Recognised Forensic Tools… FTK En. Case X-Ways Cellebrite XRY Oxgyen Accepted by the court

Recognised Forensic Tools… FTK En. Case X-Ways Cellebrite XRY Oxgyen Accepted by the court and validated in case law Non-invasive computer forensic investigative tools Cater for large volumes of data. Read FAT, NTFS, HFS, UNIX and LINUX - Proprietary Phone Systems Integrated environment allows users to perform all functions of a forensic analysis

Expand All Compound Files Archive Files ZIP RAR Complex Files OLE (Object Linking and

Expand All Compound Files Archive Files ZIP RAR Complex Files OLE (Object Linking and Embedding) Mail Boxes Outlook. pst Inbox. dbx Operating System Files Thumbs Caches Internet History

Hash All File Streams “This is a small text file. ” MD 5 (Message

Hash All File Streams “This is a small text file. ” MD 5 (Message Digest 5) Generates a unique 128 Bit value for each file or data stream: Example MD 5 Hashes. MD 5 = a 08 a 8 cf 89436 f 18 ea 8084817357 a 59 c 1 MD 5 = 271979 ddf 56 c 38805 b 7562046984 fe 40 An MD 5 Hash can be used to: Identify Files to be ignored (OS Files). Identify Files of importance (Contraband Files). “This is a small text file”

File Signature Analysis Check file header to determine if file has the correct extension

File Signature Analysis Check file header to determine if file has the correct extension Header Extension Type Result 4 d 5 a 90. . exe. dll. com Executable Match ff d 8 ff e 0. . vxd JPEG Mismatch **** . txt TEXT Unknown Highlight files with mismatch for manual checking

Entropy Test Can identify files that may be encrypted or compressed An automated frequency

Entropy Test Can identify files that may be encrypted or compressed An automated frequency analysis algorithm is used to determine if file content is encrypted Files identified are then exported from the image and transferred to specialist decryption software

Generate Index Generate an index of all strings of characters in the disk image

Generate Index Generate an index of all strings of characters in the disk image Speed up subsequent searches of suspect image Index can be used as a dictionary for password cracking

Forensics and Data relating to a “suspect” Meticulous records need to be made n

Forensics and Data relating to a “suspect” Meticulous records need to be made n ACPO guidelines must be upheld n – need to show evidence of this in court n Need to explain to jury what it all means – Essential Role: Expert Witness

“Fault Tolerant” n “A computer system or component designed so that, – in the

“Fault Tolerant” n “A computer system or component designed so that, – in the event that a component fails – a backup component or procedure can immediately take its place – with no loss of service” – https: //www. youtube. com/watch? v=P 7 g. Xm. Kd 4 Cck

Fault Tolerance and Computer Systems All about availability n Any organisation now dependent on

Fault Tolerance and Computer Systems All about availability n Any organisation now dependent on digital data n Power cut… people stop work… most of what they do involves a computer n Good fault tolerance is about minimising the chances of this happening… n

Fault Tolerance role of the Network Operating System Each important hardware component on the

Fault Tolerance role of the Network Operating System Each important hardware component on the network should have a backup that can take over in the event of a failure n It should, therefore n – detect failures – enable a backup to automatically take over when the fault is detected. . .

Achieving Fault Tolerance n ONE APPROACH… – carefully written software » software detects failure

Achieving Fault Tolerance n ONE APPROACH… – carefully written software » software detects failure of other software » takes evasive action in real time – hardware has an embedded system that: » detects failure » rapidly swaps alternative hardware into action n Makes sense for the operating system to do all of this… – detects both hardware and software failure » restarts program(s) » swaps in alternative pre-wired hardware

Concept of Data “Mirroring” n Problem with periodic backup: – data copied the previous

Concept of Data “Mirroring” n Problem with periodic backup: – data copied the previous night – what if the system hard disk goes kaput in the middle of the next day? n Copy of all data should additionally be stored “shorter term” on further media – easiest way is to have another disk in reserve – everything copied to system disk also copied to mirror

Disk Mirroring n n Increases boot/system disk fault tolerance under most conditions In its

Disk Mirroring n n Increases boot/system disk fault tolerance under most conditions In its simplest form: – – n all data held on one disk: second disk is an exact copy of the first Disk A Writes data to A Disk controller Writes same data to B When anything is written to disk… – written simultaneously to both disks Disk B

Where even Mirroring alone is not enough… n If the system crashes and will

Where even Mirroring alone is not enough… n If the system crashes and will not reboot… – operating system doesn’t get reloaded – therefore the mirror never gets activated » and copied files cannot be read…

Fault Tolerance and Re-boot n If a system crashes and/or is rebooted… – availability

Fault Tolerance and Re-boot n If a system crashes and/or is rebooted… – availability is temporarily lost Needs to be a reserve system (backup server) that will perform that system’s functions in the meantime n Network Operating system needs to synchronise processes across systems to enable this to take place… n

The Backup Server Essential for 100% availability n Should be configured as a replacement

The Backup Server Essential for 100% availability n Should be configured as a replacement for the main server n – also needs to be a domain controller – must also have a copy of the users database, regularly synchronised with the main domain controller – also configured to be able to log users onto the network

Keeping Servers Cool! Servers work hard (especially the disks…) n CPUs can get hot

Keeping Servers Cool! Servers work hard (especially the disks…) n CPUs can get hot n – will reduce MTBF of components n Need good ventilation at all times…

Minimising Effects of Power Failure n Power failure can ruin hardware – mains spikes

Minimising Effects of Power Failure n Power failure can ruin hardware – mains spikes can overheat components – sudden lack of power will lose data currently being processed n Best to protect all hardware: – bottom line - surge preventer – better: UPS (uninterruptible power supply)

The UPS n Battery packs that can provide mains voltage after a power cut

The UPS n Battery packs that can provide mains voltage after a power cut – for a few minutes (cheap but effective) – or half an hour (expensive, less down time) n n NOS needs to make sure it automatically cuts in when voltage drops sharply Power continuation must include the backup domain controller, so synchronisation can occur – procedure of “graceful degradation” » allows processing to go to completion » allows new system settings to be written

The Fault Tolerant Network Operating System A Fault Tolerant system needs to have good

The Fault Tolerant Network Operating System A Fault Tolerant system needs to have good control of hardware, backup hardware and software n The NOS, and those who configure it, need to use fault tolerance effectively so an organisational network will n – keep going… (accessibility) – do what is expected… (reliability, stability)

Business Continuity… n More and more businesses use solely digital systems – saved data

Business Continuity… n More and more businesses use solely digital systems – saved data very precious! – if not looked after and no copy taken… » no plan B if data destroyed e. g. by flooding » no data means no business! n Need also to save data to a secure, but different location as part of Business Continuity Planning (BCP)