Towards a peertopeer file backup system Alessandro Duminuco

Towards a peer-to-peer file backup system Alessandro Duminuco EURECOM Sophia Antipolis - France

Outline Motivation Challenges Hints on Solutions Conclusion 26/11/2020 - EURECOM RESEARCH -p 2

Motivation Peer-to-peer Systems Once upon a time the client-server pattern When the number of clients increases. . . you buy a bigger server. . . and bigger! 26/11/2020 - EURECOM RESEARCH -p 3

Motivation Peer-to-peer Systems In P 2 P systems, peers collaborate sharing resources When the number of clients increases. . . the resources increase as well! 26/11/2020 - EURECOM RESEARCH -p 4

Motivation Peer-to-peer Systems § Peers can share all kinds of resources: Ø Bandwidth: Content distribution, video streaming, Vo. IP. Ø CPU: Grid Computing, Online Gaming. Ø Storage: File Sharing. § What do these applications have in common? Ø Central Solutions are costly. Ø Peers can provide the needed resources at low price: FCPUs are mostly idle. FDisks are underutilized. FThey have plenty of bandwidth. . perhaps 26/11/2020 - EURECOM RESEARCH -p 5

P 2 P Backup File Backup System Motivation § Does File Backup fit the P 2 P model? (Is it worth working on this? ) § Of course I believe so, because: § Everybody is looking for cheap backup (ex. photos) § Central solutions can be afforded only by companies. § Users’ PCs are equipped with huge disks. § Microsoft supported me: there must be a reason! 26/11/2020 - EURECOM RESEARCH -p 6

Outline Motivation Challenges Hints on Solutions Conclusion 26/11/2020 - EURECOM RESEARCH -p 7

Challenges Life is not easy § Unfortunately. . . Ø. . . peers are not always online. Ø. . . peers can abandon the system. Ø. . . peers can be evil and deliberately delete data. § We have to cope with all this and. . . Ø. . . with limited bandwidth, Ø. . . without wasting too much storage, Ø. . . without losing data! 26/11/2020 - EURECOM RESEARCH -p 8

The basic solution: Replication Challenges File § With 4 replicas, even if 3 peers are offline we still have the file. § Every file consumes storage for 4 times its size!! 26/11/2020 - EURECOM RESEARCH -p 9

Challenges A better solution: coding Dissemination Coding k k+h File Fragments § Any k fragments are sufficient to reconstruct the file: we can sustain any h losses. § Every file consumes storage for (k+h)/k times its size. § If k=6 and h=3, (k+h)/k=1. 5 …. Instead of 4!! 26/11/2020 - EURECOM RESEARCH - p 10

Challenges Repair Process § Initial redundancy can only deal with temporary disconnections of peers. § When peers abandon the overlay, we need to introduce new fragments (Repair). Abandons Repairs Fragments § Repair process: replace abandons by repairs. Challenge 1: How to recognize abandons? Can we do repairs smoothly? 26/11/2020 - EURECOM RESEARCH - p 11

Challenges Coding is cool but… § A repair in replication means copying one replica from one peer to another. § In coding… k Fragments …a repair means combining k fragments into a new one. § We create a single fragment and we transfer the whole file!! Challenge 2: Can we afford it? 26/11/2020 - EURECOM RESEARCH - p 12

Outline Motivation Challenges Hints on Solutions Conclusion 26/11/2020 - EURECOM RESEARCH - p 13

Hints on Solutions Statistical estimator § We observe reconnections and disconnections and we decide the rate of the repairs. Available Fragments Repairs Reconnections n Disconnections Lost Fragments Unavailable Fragments Repairs balance the abandons in a smooth way 26/11/2020 - EURECOM RESEARCH - p 14

Hints on Solutions Hierarchical Codes § We proposed a class of linear codes where the number of fragments to read for a repair is in average lower than k. k d<k Fragments § The price to pay is a higher number of repairs, which results however in a low communication cost. We can use coding with affordable bandwidth needs 26/11/2020 - EURECOM RESEARCH - p 15

Outline Motivation Challenges Hints on Solutions Conclusion 26/11/2020 - EURECOM RESEARCH - p 16

Conclusion § I gave a lot of questions and almost no answers. § The objective was attract your interest on my work. If I succeeded: Alessandro Duminuco, Ernst Biersack, Taoufik En-Najjary Proactive Replication in Distributed Storage Systems Using Machine Availability Estimation. Co. NEXT, December 2007, New York - USA Alessandro Duminuco, Ernst Biersack Hierarchical Codes: How to Make Erasure Codes Attractive for Peer-to-Peer Storage Systems. P 2 P, September 2008, Aachen - Germany 26/11/2020 - EURECOM RESEARCH - p 17

Questions ? 26/11/2020 - EURECOM RESEARCH - p 18