The EigenTrust Algorithm for Reputation Management in P2P ...

Edge computing (1) Content Distribution Networks Chen Qian Department of Computer Science and Engineering [email protected] https://users.soe.ucsc.edu/~qian/ Algorithmic Nuggets in Content Delivery Bruce M. Maggs Ramesh K. Sitaraman Overview Background Representative research Conclusion 3

Web caches (proxy server) goal: satisfy client request without involving origin server user sets browser: Web accesses via cache browser sends all HTTP requests to cache object in cache: cache returns object else cache requests object from origin server, then returns object to client proxy

HT TP client HT TP r req ues server t esp ons

e req TP T H st que e r TP HT nse spo e r TP HT

origin server st ue TP HT p res s on e client

origin server Application Layer 4 More about Web caching cache acts as both client and server server for original requesting client

client to origin server typically cache is installed by ISP (university, company, residential ISP) why Web caching? reduce response time for client request reduce traffic on an institutions access link When is cache not good? Every client of the ISP requests different content. Waste time on visiting

cache server Application Layer 5 Background Content delivery network (CDN) A geographically distributed network of proxy servers and their data centers. Distribute service spatially relative to end-users e.g. Service for DNS query 6

Background Top-three objectives of CDN High reliability Fast and consistent performance Low operating cost Representative research on CDN 7 Global load balancing

Load balancing within a single cluster of servers Bloom filters for CDN Overlay routing Leader election and consensus Overview Background Representative research Conclusion 8 Representative research on CDN Global load balancing Load balancing within a single cluster of servers Bloom filters for CDN Leader election and consensus

9 Global Load Balancing Purpose: Map clients to the server clusters of the CDN. Clusters assignments are made at the granularity of map units. e.g. <1.2.3.4/24, video> Question: How to assign each map unit to a server cluster 10 Global Load Balancing Preference

Each map unit has preferences for clusters, higher preference indicates better predicted performance. Each server cluster has preferences regarding which map units it would ints like to serve. stra Constraints& Meet ca es c n e r refe

p y sf Sati ty paci con Each map unit is associated with a demand . Each cluster has a notion of capacity . 11 Global Load Balancing Stable allocations

12 Blocking pairs: prefers over its current partner & prefers over its current partner Stable: There are no blocking pairs in the allocations 1 :( 2 ,1 , 3 ) 1 :( 2 , 1 , 3 ) 2 :( 2 , 1 , 3 ) 2 :( 1 , 2 , 3 ) 3 :( 2 , 3 ,1 )

3 :( 1 , 3 , 2) Gale-Shapley algorithm Gale-Shapley algorithm is a distributed algorithm to find a stable allocations. (Propose-And-Reject algorithm) 13 man(who proposes)-optimal map-unit-optimal Some Limitations Unequal number of map units and clusters More map units than clusters

Partial preference lists Tens of millions of map units VS Thousands of clusters Rank for each map unit the top dozen clusters that are likely to provide the best performance Modeling integral demands and capacities A server cluster cannot be accurately modeled as a single resource with a single number capacity 14 A Survey of the Stable Marriage Problem and Its Variants Resource Trees Bps: the rate at which data can be sent out of the cluster modeled. A

Fps: the capacity of non-network serve resources such as the 50 Bps processor, memory and disk. Violation B 25 Fps Video Apps Web C 30 Fps 5 Fps & 20 Bps

25 Fps 15 30 Fps D 40 Fps E 1. 20 units of demand from a video map, each unit requires 0.25 Fps and 1 Bps. (5 Fps & 20 Bps) 30 Fps 26 Fps & 6.5Bps 4 Fps 2. 26 units of demand from application map, each unit requires 1 Fps and 0.25 Bps. (26 Fps & 6.5 Bps)

Resource Trees If cluster has a higher preference for map units with application A 50 Bps traffic than video traffic Violation B 25 Fps Video Apps Web

C 30 Fps 5 Fps & 20 Bps 25 Fps 16 30 Fps D 40 Fps E 30 Fps 26 Fps & 6.5Bps 4 Fps Evict a lower preference map unit

e.g. 4 units of video demand (1 Fps) are evicted. Implementation Challenges Complexity and scale Tens of millions of map units Thousands of clusters Over a dozen traffic classes Time to solve Map unit assignment must be recomputed every 10 to 30 seconds Demand and capacity estimation

Incremental and persistent allocation 17 Representative research on CDN Global load balancing Load balancing within a single cluster of servers Bloom filters for CDN Leader election and consensus 18 Problem Statement In a traditional hash table, objects from a universe are mapped to a set of buckets . In CDN, an object is a file such as a JPEG image or HTML page; a bucket is the cache of a distinct web server. Naive method: Use hash functions to directly

map objects to buckets. If the servers fail? 19 Problem Statement Solution 1: Simply remap objects in the lost bucket to another bucket One bucket stores double the expected load Solution 2:

20 in h s a H t n g te sexisting Renumber the buckets and rehash the i s n

Co elements using a new hash function Many objects will have to be transferred between buckets. Consistent Hashing Server Object 21 Each object is mapped to the next bucket that appears in clockwise order on the unit circle. Consistent Hashing Improvements Map each bucket to multiple locations (instances)

on the unit circle to improve the balance When a server fails 22 All of the corresponding buckets instances are removed from the unit circle The objects that were in the buckets are remapped to other buckets. Consistent Hashing Popular objects It is not possible for a single server within a cluster to

satisfy all of the requests for a popular object Naive method: map a popular object to the next k servers that appear in clockwise order Problem: If two popular objects happen to hash to nearby positions, the buckets that they map to will highly overlap CDN approach 23 Use a separate mapping of the buckets for each popular object Representative research on CDN Global load balancing

Load balancing within a single cluster of servers Bloom filters for CDN Leader election and consensus 25 Bloom filters for CDN Bloom filters are useful in two different contexts Content summarization Content filtering Content summarization

26 Use Bloom filters to succinctly store the set of objects stored in a CDN servers cache Use counting Bloom filters to support elements update (deletions and insertions) Content Filtering Use Bloom filters to determine what objects to cache in the first place 74% of the roughly 400 million Motivation objects in cache were accessed only once (one-hit-wonders) 90% were accessed less than four

times No need to cache one-hit-wonders. 27 Content Filtering Cache-on-second-hit rule Use Bloom filters to store accessed objects Server checks Bloom filters to see whether the object has been accessed before Server caches the objects have been accessed before

False positives 28 The probability of false positives increases with more objects are added to a Bloom filter. Use two Bloom filters to circumvent the problem Content Filtering New objects Primary Bloom filter It reaches a threshold for maximum number of objects Then new objects Secondary Bloom filter

Check both the primary and secondary Bloom filters to see if the object has been accessed in the recent past 29 Content Filtering Benefits Byte hit rates increased when cache filtering was turned on 30 Content Filtering Benefits Not having to store the one-hit-wonders in cache reduces the disk writes by nearly one-half 31 Representative research on CDN

Global load balancing Load balancing within a single cluster of servers Bloom filters for CDN Overlay routing Leader election and consensus 33 Overview Background Representative research Conclusion 34 Conclusion This paper explores some research problems in CDN.

The purpose of the paper is to illustrate 35 How research influenced the design of CDN How the system-building challenges inspired more research in CDN Thank You Chen Qian [email protected] https://users.soe.ucsc.edu/~qian/

Recently Viewed Presentations

  • Mobilnost sudionika u Republiku Irsku Dublin 21.10.-27.10.2012. Mobilnost

    Mobilnost sudionika u Republiku Irsku Dublin 21.10.-27.10.2012. Mobilnost

    Mobilnost sudionika u Republiku Irsku Dublin 21.10.-27.10.2012. Mobilnost u Republiku Irsku Mobility to Republic Ireland for acquiring new knowledge through cooperation with staff in vocational training and experience in European firms in the field mechanical engineering and electronics and automation...
  • Joseph G. Luther Elementary School

    Joseph G. Luther Elementary School

    Introductions. Special Education. Kerri Boardman, Room 1 . Kelly Raleigh, Room 1 . Crystal St. Pierre, Room 1. Sandra Kozatek, Room 5 . Kelsey Martins, Room 5
  • A katalízis jelentősége, magyarok a katalízisben

    A katalízis jelentősége, magyarok a katalízisben

    A katalízis jelentősége, magyarok a katalízisben, katalízis a Műegyetemen Dr Tungler Antal egyetemi tanár MTA IKI BME KKFT 2009 Kémiai Nobel díjak 97 kémiai Nobel díjból 10 jutott a katalízisre illetve ahhoz szorosan kapcsolódó felfedezésre Magyarok a világ katalízis kutatásában...
  • PowerPoint Template

    PowerPoint Template

    Phânloại tri thức [1] Tri thức sự kiện khẳng định về một sự kiện, hiện tượng hay một khái niệm nào đó trong một hoàn cảnh không gian hoặc thời gian nhất định: địnhlýtoánhọc, địnhluậtvậtlý, … [2] Tri thức thủ...
  • Unified Communications, Collaboration, and Cloud S o l

    Unified Communications, Collaboration, and Cloud S o l

    Customers not ready to use Lync Server for voice, but who want to take advantage of Lync Server conferencing to lower costs and improve capabilities. Issues Resolved: Your customer can keep the existing PBX system, while adding Lync Server 2010...
  • Vector addition

    Vector addition

    Determine the resultant velocity of the riverboat (velocity with respect to the shore…how would it look to an observer?). c. If the river is 71.0 m wide, then determine the time required for the boat to cross the river. d....
  • Requirements and User Stories - School of Computing and ...

    Requirements and User Stories - School of Computing and ...

    Professor Alex Roque. Sequential Requirements. In Sequential product development, we attempt to provide a specific detail description of what the software needs to do. Once this requirement is detailed, we try not to od any changes, because any change will...
  • Major Published Clinical Trials in AKI: What do

    Major Published Clinical Trials in AKI: What do

    McGill University Health Centre. What does "Clinical Trials in AKI" mean? ... Care Med, 2008. Highlight: Poor quality evidence, heterogeneity. Timing and dose ... found difference in mortal/ESRD with pre-op Epo, single cnetre, other problems, but point of biomarkers.... AKI...