Fragmentation in Large Object Repositories课件.ppt

Experimental Setup Single storage node Compared filesystem, database NTFS on Windows Server 2003 R2 SQL Server 2005 beta Repeatedly update (free, reallocate) objects Randomly chose sizes, objects to update Unrealistic, easy to understand Measured throughput, fragmentation 精品文档 Reasoning about time Existing metrics Wall clock time: Requires trace to be meaningful, cannot compare different workloads Updates per volume: Coupled to volume size Storage Age: Average number of updates per object 精品文档 Read performance Clean system SQL good small object performance (inexpensive opens) NTFS significantly faster with objects 1MB SQL degraded quickly NTFS small object performance was low, but constant 0 2 4 6 8 10 12 NTFS SQL NTFS SQL Read Throughput (MB/s) 0 2 4 Updates per object 256 KB Objects 1 MB Objects 精品文档 10MB object fragmentation 0 5 10 15 20 25 30 35 40 0 1 2 3 4 5 6 7 8 9 10 Storage Age Fragments/object SQL Server NTFS NTFS approaching asymptote SQL Server degrades linearly No BLOB defragmenter 精品文档 Rules of Thumb Classic pitfalls Low free space ( 10%) Repeated allocation and deallocation (High storage age) One new problem Small volumes ( 100-1000x object size) Implicit tuning knobs Size of write requests 精品文档 Append is expensive! Neither system can take advantage of final object size during allocation Both API’s provide “append” Leave gaps for future appends Place objects without knowing length Observe same behavior with single and random object sizes 精品文档 Conclusions Get/put storage is important in practice Storage age Metric for comparing implementations and workloads Fragmentation behaviors vary significantly Append leads to poor layout 精品文档 ----BACKUP SLIDES---- 精品文档 Theory vs. Practice Theory focuses on contiguous layout of objects of known size Objects that are allocated in groups are freed in groups Good allocation algorithms exploit this Generally ignored for average case results Leads to pathological behavior in some cases 精品文档 Small objects / Large