欧美精品一二区,性欧美一级,国产免费一区成人漫画,草久久久久,欧美性猛交ⅹxxx乱大交免费,欧美精品另类,香蕉视频免费播放

《google文件系統(tǒng)》PPT課件.ppt

上傳人:za****8 文檔編號:15183844 上傳時(shí)間:2020-08-05 格式:PPT 頁數(shù):24 大?。?23KB
收藏 版權(quán)申訴 舉報(bào) 下載
《google文件系統(tǒng)》PPT課件.ppt_第1頁
第1頁 / 共24頁
《google文件系統(tǒng)》PPT課件.ppt_第2頁
第2頁 / 共24頁
《google文件系統(tǒng)》PPT課件.ppt_第3頁
第3頁 / 共24頁

下載文檔到電腦,查找使用更方便

9.9 積分

下載資源

還剩頁未讀,繼續(xù)閱讀

資源描述:

《《google文件系統(tǒng)》PPT課件.ppt》由會(huì)員分享,可在線閱讀,更多相關(guān)《《google文件系統(tǒng)》PPT課件.ppt(24頁珍藏版)》請?jiān)谘b配圖網(wǎng)上搜索。

1、GFS: The Google File System,Brad Karp UCL Computer Science,CS Z03 / 4030 30th October, 2006,2,Motivating Application: Google,Crawl the whole web Store it all on “one big disk” Process users searches on “one big CPU” More storage, CPU required than one PC can offer Custom parallel supercomputer: expe

2、nsive (so much so not really available today),3,Cluster of PCs as Supercomputer,Lots of cheap PCs, each with disk and CPU High aggregate storage capacity Spread search processing across many CPUs How to share data among PCs? Ivy: shared virtual memory Fine-grained, relatively strong consistency at l

3、oad/store level Fault tolerance? NFS: share fs from one server, many clients Goal: mimic original UNIX local fs semantics Compromise: close-to-open consistency (performance) Fault tolerance?,4,Google Platform Characteristics,100s to 1000s of PCs in cluster Cheap, commodity parts in PCs Many modes of

4、 failure for each PC: App bugs, OS bugs Human error Disk failure, memory failure, net failure, power supply failure Connector failure Monitoring, fault tolerance, auto-recovery essential,5,Google File System: Design Criteria,Detect, tolerate, recover from failures automatically Large files, = 100 MB

5、 in size Large, streaming reads (= 1 MB in size) Read once Large, sequential writes that append Write once Concurrent appends by multiple clients (e.g., producer-consumer queues) Want atomicity for appends without synchronization overhead among clients,6,GFS: Architecture,One master server (state re

6、plicated on backups) Many chunk servers (100s 1000s) Spread across racks; intra-rack b/w greater than inter-rack Chunk: 64 MB portion of file, identified by 64-bit, globally unique ID Many clients accessing same and different files stored on same cluster,7,GFS: Architecture (2),8,Master Server,Holds

7、 all metadata: Namespace (directory hierarchy) Access control information (per-file) Mapping from files to chunks Current locations of chunks (chunkservers) Manages chunk leases to chunkservers Garbage collects orphaned chunks Migrates chunks between chunkservers,9,Chunkserver,Stores 64 MB file chun

8、ks on local disk using standard Linux filesystem, each with version number and checksum Read/write requests specify chunk handle and byte range Chunks replicated on configurable number of chunkservers (default: 3) No caching of file data (beyond standard Linux buffer cache),10,Client,Issues control

9、(metadata) requests to master server Issues data requests directly to chunkservers Caches metadata Does no caching of data No consistency difficulties among clients Streaming reads (read once) and append writes (write once) dont benefit much from caching at client,11,Client API,Is GFS a filesystem i

10、n traditional sense? Implemented in kernel, under vnode layer? Mimics UNIX semantics? No; a library apps can link in for storage access API: open, delete, read, write (as expected) snapshot: quickly create copy of file append: at least once, possibly with gaps and/or inconsistencies among clients,12

11、,Client Read,Client sends master: read(file name, chunk index) Masters reply: chunk ID, chunk version number, locations of replicas Client sends “closest” chunkserver w/replica: read(chunk ID, byte range) “Closest” determined by IP address on simple rack-based network topology Chunkserver replies wi

12、th data,13,Client Write,Some chunkserver is primary for each chunk Master grants lease to primary (typically for 60 sec.) Leases renewed using periodic heartbeat messages between master and chunkservers Client asks server for primary and secondary replicas for each chunk Client sends data to replica

13、s in daisy chain Pipelined: each replica forwards as it receives Takes advantage of full-duplex Ethernet links,14,Client Write (2),All replicas acknowledge data write to client Client sends write request to primary Primary assigns serial number to write request, providing ordering Primary forwards w

14、rite request with same serial number to secondaries Secondaries all reply to primary after completing write Primary replies to client,15,Client Write (3),16,GFS: Consistency Model,Changes to namespace (i.e., metadata) are atomic Done by single master server! Master uses log to define global total or

15、der of namespace-changing operations Data changes more complicated Consistent: file region all clients see as same, regardless of replicas they read from Defined: after data mutation, file region that is consistent, and all clients see that entire mutation,17,GFS: Data Mutation Consistency,Record ap

16、pend completes at least once, at offset of GFS choosing Apps must cope with Record Append semantics,18,Applications andRecord Append Semantics,Applications should include checksums in records they write using Record Append Reader can identify padding / record fragments using checksums If application

17、 cannot tolerate duplicated records, should include unique ID in record Reader can use unique IDs to filter duplicates,19,Logging at Master,Master has all metadata information Lose it, and youve lost the filesystem! Master logs all client requests to disk sequentially Replicates log entries to remot

18、e backup servers Only replies to client after log entries safe on disk on self and backups!,20,Chunk Leases and Version Numbers,If no outstanding lease when client requests write, master grants new one Chunks have version numbers Stored on disk at master and chunkservers Each time master grants new

19、lease, increments version, informs all replicas Master can revoke leases e.g., when client requests rename or snapshot of file,21,What If the Master Reboots?,Replays log from disk Recovers namespace (directory) information Recovers file-to-chunk-ID mapping Asks chunkservers which chunks they hold Re

20、covers chunk-ID-to-chunkserver mapping If chunk server has older chunk, its stale Chunk server down at lease renewal If chunk server has newer chunk, adopt its version number Master may have failed while granting lease,22,What if Chunkserver Fails?,Master notices missing heartbeats Master decrements

21、 count of replicas for all chunks on dead chunkserver Master re-replicates chunks missing replicas in background Highest priority for chunks missing greatest number of replicas,23,File Deletion,When client deletes file: Master records deletion in its log File renamed to hidden name including deletio

22、n timestamp Master scans file namespace in background: Removes files with such names if deleted for longer than 3 days (configurable) In-memory metadata erased Master scans chunk namespace in background: Removes unreferenced chunks from chunkservers,24,GFS: Summary,Success: used actively by Google t

23、o support search service and other applications Availability and recoverability on cheap hardware High throughput by decoupling control and data Supports massive data sets and concurrent appends Semantics not transparent to apps Must verify file contents to avoid inconsistent regions, repeated appends (at-least-once semantics) Performance not good for all apps Assumes read-once, write-once workload (no client caching!),

展開閱讀全文
溫馨提示:
1: 本站所有資源如無特殊說明,都需要本地電腦安裝OFFICE2007和PDF閱讀器。圖紙軟件為CAD,CAXA,PROE,UG,SolidWorks等.壓縮文件請下載最新的WinRAR軟件解壓。
2: 本站的文檔不包含任何第三方提供的附件圖紙等,如果需要附件,請聯(lián)系上傳者。文件的所有權(quán)益歸上傳用戶所有。
3.本站RAR壓縮包中若帶圖紙,網(wǎng)頁內(nèi)容里面會(huì)有圖紙預(yù)覽,若沒有圖紙預(yù)覽就沒有圖紙。
4. 未經(jīng)權(quán)益所有人同意不得將文件中的內(nèi)容挪作商業(yè)或盈利用途。
5. 裝配圖網(wǎng)僅提供信息存儲空間,僅對用戶上傳內(nèi)容的表現(xiàn)方式做保護(hù)處理,對用戶上傳分享的文檔內(nèi)容本身不做任何修改或編輯,并不能對任何下載內(nèi)容負(fù)責(zé)。
6. 下載文件中如有侵權(quán)或不適當(dāng)內(nèi)容,請與我們聯(lián)系,我們立即糾正。
7. 本站不保證下載資源的準(zhǔn)確性、安全性和完整性, 同時(shí)也不承擔(dān)用戶因使用這些下載資源對自己和他人造成任何形式的傷害或損失。

相關(guān)資源

更多
正為您匹配相似的精品文檔
關(guān)于我們 - 網(wǎng)站聲明 - 網(wǎng)站地圖 - 資源地圖 - 友情鏈接 - 網(wǎng)站客服 - 聯(lián)系我們

copyright@ 2023-2025  zhuangpeitu.com 裝配圖網(wǎng)版權(quán)所有   聯(lián)系電話:18123376007

備案號:ICP2024067431-1 川公網(wǎng)安備51140202000466號


本站為文檔C2C交易模式,即用戶上傳的文檔直接被用戶下載,本站只是中間服務(wù)平臺,本站所有文檔下載所得的收益歸上傳人(含作者)所有。裝配圖網(wǎng)僅提供信息存儲空間,僅對用戶上傳內(nèi)容的表現(xiàn)方式做保護(hù)處理,對上載內(nèi)容本身不做任何修改或編輯。若文檔所含內(nèi)容侵犯了您的版權(quán)或隱私,請立即通知裝配圖網(wǎng),我們立即給予刪除!