更灵活,更易用 — ZFS storage 统一存储介绍
- 格式:pdf
- 大小:3.87 MB
- 文档页数:23
zfs使用心得【原创实用版3篇】目录(篇1)1. ZFS 简介2. ZFS 的特点3. ZFS 的使用方法4. ZFS 的优点5. ZFS 的缺点6. 总结正文(篇1)1. ZFS 简介ZFS(Zettabyte File System)是一种高性能、可扩展的文件系统,主要用于存储大量数据。
ZFS 是由 Sun Microsystems 公司开发的,现在由 OpenZFS 社区维护。
ZFS 的特点是数据完整性高、可扩展性强、性能优秀,因此在企业级存储领域得到了广泛应用。
2. ZFS 的特点ZFS 具有以下几个显著特点:(1)数据完整性:ZFS 支持数据校验和,可以检测和修复数据错误,保证数据的完整性。
(2)可扩展性:ZFS 支持无限大的文件系统,可以存储大量数据。
(3)高性能:ZFS 具有高效的磁盘 I/O 调度算法,可以提高系统的磁盘吞吐量。
3. ZFS 的使用方法要使用 ZFS,首先需要在操作系统中安装 ZFS 模块。
在 Linux 系统中,可以通过安装 zfs-tools 软件包来获得 ZFS 支持。
在 FreeBSD 系统中,ZFS 已经被集成到内核中,无需额外安装。
目录(篇2)1.ZFS 简介2.ZFS 的主要特点3.ZFS 的实际应用4.ZFS 的未来发展前景正文(篇2)【ZFS 简介】ZFS(Zettabyte File System)是一种用于管理大规模数据的文件系统,最早由 Sun Microsystems 公司开发。
ZFS 的主要特点是数据可靠性高、可扩展性强以及性能优越。
它能够提供大量的数据存储和处理能力,因此非常适合用于企业级服务器和大型数据中心。
【ZFS 的主要特点】1.数据可靠性:ZFS 采用了一种名为“数据完整性”的技术,可以确保数据的完整性和可靠性。
即使在硬盘故障或者意外断电的情况下,ZFS 也能够自动修复数据,确保数据的安全。
2.可扩展性:ZFS 可以轻松地扩展到数百 TB 甚至数 PB 的存储空间,满足大规模数据的存储需求。
统一存储和分布式存储-回复统一存储和分布式存储:解决数据存储难题的两个方法随着云计算和大数据的迅猛发展,数据存储成为了一个非常重要的问题。
为了解决大规模数据存储的需求,统一存储和分布式存储成为了两种备受关注的方法。
本文将从介绍统一存储和分布式存储的概念开始,并逐步深入探讨这两者的原理、优势以及应用场景。
一、统一存储的概念及原理统一存储(Unified Storage),顾名思义,是指将不同类型的存储设备或技术整合到一个统一的存储系统中进行管理和使用。
传统的数据存储往往需要使用不同的设备或技术来处理不同类型的存储需求,例如,文件存储、块级存储和对象存储等。
而统一存储的出现使得企业可以使用一个系统来管理和操作不同类型的存储需求,极大地简化了存储管理的复杂性。
统一存储的原理主要包括三个方面:存储虚拟化、存储管理和存储协议。
首先,通过存储虚拟化技术,统一存储将不同类型的存储设备虚拟化为一个逻辑存储池,隐藏了设备的具体细节,统一了不同设备的管理接口。
其次,统一存储通过存储管理技术实现了数据的统一管理,包括数据的分配、备份、快照以及容灾等功能。
最后,通过支持各种存储协议,如NFS、CIFS、iSCSI、FC等,统一存储可以满足不同应用的存储需求。
二、统一存储的优势统一存储相较于传统的存储方式具有以下几个明显的优势:1. 简化管理:统一存储通过将各种存储设备整合到一个系统中,简化了存储管理的复杂性。
管理员可以通过一个界面进行一体化管理,提高了工作效率。
2. 降低成本:传统的存储方式需要使用不同的设备和技术,而统一存储可以将不同类型的存储需求整合到一个系统中,避免了重复投资,并提高了设备的利用率,从而降低了存储成本。
3. 提高可靠性:统一存储支持多种数据保护功能,包括备份、快照和容灾等,可以有效地保护数据的安全性和可靠性。
4. 提升性能:统一存储通过对存储资源进行统一管理和调度,可以提高存储的性能和响应速度,满足高速读写的需求。
Oracle ZFS Storage Appliance is flexible multiprotocol enterprise storage that delivers high performance, unmatched efficiency, and superior manageability for many types of workloads. For Oracle Database business intelligence/data warehousing (BIDW) workloads, Oracle ZFS Storage Appliance provides compelling performance and availability benefits. In addition, unique Oracle hardware and software co-engineering delivers performance, manageability, and efficiency advantages unavailable from other third-party systems. Using Oracle storage with Oracle Database directly empowers database administrators (DBAs) to efficiently manage and optimize the entire stack. Furthermore, by being co-engineered with Oracle Public Cloud, Oracle ZFS Storage Appliance offers a built-in cloud gateway for on-demand access to seamlessly take advantage of cloud capacity and economics.The Correct Tools for the JobOracle Exadata is the ultimate platform for Oracle Database from a performance and availability perspective. However, if you have consolidated workloads, heterogeneous vendors to support, flexible performance or availability requirements, or stringent cost constraints, an alternate solution is sometimes required. Oracle ZFS StorageAppliance, next-generation engineered storage, is just that solution due to its unique coengineering with Oracle software. This solution delivers special integrations that simplify management, reduce risk, and drive operational efficiencies.Like Oracle Exadata, Oracle ZFS Storage Appliance supports Oracle Database’s Hybrid Columnar Compression, a feature available only with Oracle Database 11g Release 2 and higher releases, when it is used exclusively in conjunction with Oracle storage systems. With multiple compression levels suitable for query and archival workload profiles, Hybrid Columnar Compression significantly enhances storage capacity utilization efficiency in many read-intensive Oracle Database data warehousing workloads. Because Hybrid Columnar Compression moves compressed data from the host to the storage, it actually uses bandwidth more efficiently and can result in higher throughput performance. In fact, customers accelerate critical BIDW queries an average of 8x. Additionally, Hybrid Columnar Compression can be used in conjunction with the Automatic Data Optimization feature of Oracle Database 12c to dynamically manage compression and storage tiering. This pairing helps you efficiently optimize partitioning for data footprint efficiency and query response performance O R A C L E O F F E R S P R O V E NS O L U T I O N S F O R O R A C L E D A T A B A S E P R I M A R Y S T O R A G E U S I N G T H EO R A C L E Z F S S T O R A G E A P P L I A N C EF O R B I D W W O R K L O A D S.C O N N E C T W I T H U S /oracle /oracle /oracle F O R M O R E I N F O R M A T I O N Contact: 1.800.ORACLE1 without manual intervention. This feature can be further enhanced by movingnoncritical data to the cloud and taking advantage of on-demand capacities in thecloud. For more information, please see the “Automatic Data Optimization with OracleDatabase 12c ” white paper.Superior BIDW PerformanceBIDW storage workloads are generally characterized by large-block-size high-throughput reads in response to query commands. From a performance perspective,the key requirement is high bandwidth throughput as opposed to high transactionalfrequency (that is, MB/sec is a priority over IOPS). Oracle ZFS Storage Appliancecombines a scalable storage operating system, high-throughput system design, andintelligent caching to deliver superior performance for just this type of workload. Plus,with all-flash and hybrid flash/disk configurations available, Oracle ZFS StorageAppliance delivers high performance at a lower cost —enabling betterprice/performance compared to competitive systems. For large and growing BIDWworkloads, this price/performance advantage is critical. For further information, pleasesee the “Realizing the Superior Value and Performance of Oracle ZFS StorageAppliance ” white paper. High Availability and Data ProtectionAside from performance, high availability is required in many BIDW environments sothat queries can continue to be served in the event of component failures. Oracle ZFSStorage Appliance offers high availability through dual-controller configurations withcontroller failover mechanism, drive failure redundancy, and network redundancyfeatures. Furthermore, the ZFS file system performs end-to-end checksumming andself-healing to ensure data integrity and prevent silent data corruption. Please see the“Architectural Overview of the Oracle ZFS Storage Appliance ” white paper for furtherdetails. Additionally, when the Data Guard feature of Oracle Database, EnterpriseEdition, is used in conjunction with Oracle ZFS Storage Appliance, a fullysynchronous, duplicate instance of the database can be maintained on a redundantsystem. This offers you the ultimate in-system or site-level outage protection with farless performance impact than traditional storage-based synchronous replicationsolutions. Oracle has documented best practices for tuning Oracle ZFS StorageAppliance for optimal performance and for availability by ensuring that database hostsdo not time out and applications continue running with zero data loss and minimaldisruption time.One of the compelling features of Oracle ZFS Storage Appliance is the ability to tunesettings for optimal performance and efficiency in different workloads. While diskmirroring is available for high transactional performance, single-parity RAID can beemployed in many throughput-oriented BIDW workloads for performance and spaceefficiency benefits. Also, share settings, including ZFS Intent Log (ZIL) behavior andblock sizes, can be tuned for different workloads, and for different conditions. Forexample, a share that contains a data warehouse compressed by Hybrid ColumnarCompression might optimally use a different record size than a share that contains anuncompressed data warehouse, and yet another record size would be used for othershares that are transactional in nature. Thus, unlike most storage systems from otherH Y B R I D C O L U M N A R C O M P R E S S I O N , U N I Q U E L Y A V A I L A B L E W I T H T H E C O M B I N A T I O N O F O R A C L E D A T A B A S E A N D O R A C L E S T O R A G E , C A N P R O V I D E U P T O 50XC O M P R E S S I O N I ND A T AW A R E H O U S I N G E N V I R O N M E N T S. Oracle ZFS Storage Appliance offers compelling benefits in an Oracle Database primary storage environment: • High availability • High performance in both transactional and BIDW workloads • Reduced management complexity costs at a lower upfront cost • Unprecedented integration with Oracle Database, such as Oracle Intelligent Storage Protocol and Hybrid Columnar Compression • Proven best practices used in Oracle I T’s own data centersC O N N E C T W I T H U S /oracle/oracle/oracle F O R M O R E I N F O R M A T I O N Contact: 1.800.ORACLE1vendors, Oracle ZFS Storage Appliance can be optimally tuned for a variety of distinctOracle Database workloads. To eliminate the tuning and guesswork required tooptimize these share parameters, Oracle ZFS Storage Appliance has exclusivecoengineering with Oracle Database 12c to dynamically automate this as abackground process. This unique technology feature of Oracle ZFS StorageAppliance, Oracle Intelligent Storage Protocol, can reduce administrative steps to tunedatabase storage by up to 65 percent. Oracle Intelligent Storage Protocol sends hintsfrom Oracle 12c and higher databases to Oracle ZFS Storage Appliance to optimizecritical storage resources without manual intervention, and it can actually prioritize I/Oto further improve performance by reducing resource contention. This can enableconsolidation of diverse database workloads on a single appliance by managing andoptimizing based on the specific incoming database I/O. See the “Oracle IntelligentStorage Protocol ” data sheet for more information. These database-awaretechnologies such as Hybrid Columnar Compression and Oracle Intelligent StorageProtocol result in much faster Oracle Database BIDW workload performance.Simplified ManagementOracle ZFS Storage Appliance also offers integration with Oracle Enterprise ManagerCloud Control. In addition to having its own Oracle Enterprise Manager plug-inavailable for single-pane-of-glass visibility of multiple appliances, Oracle ZFS StorageAppliance provides Snap Clone (a feature of Oracle Cloud Management Pack forOracle Database) integration with Oracle Enterprise Manager 12c and above toprovide database-as-a-service (DBaaS) capabilities. This is useful because it allowsadministrators to make space-efficient thin copies of entire data warehouses throughthe Oracle Enterprise Manager interface. When they use snap cloning, the copyoperation can occur much faster and occupy far less storage capacity than other copyoptions supported by most storage vendors. Snap Clone can be used in conjunctionwith the DBaaS schema provisioning functionality for rapid, low-storage provisioning ofdata marts for targeted analytical purposes.Further information on DBaaS and Oracle ZFS Storage Appliance integration usage isavailable in the “Delivering Database as a Service (DBaaS) using Oracle EnterpriseManager 12c ” white paper. Unique with Oracle ZFS Storage Appliance, thisfunctionality also can be paired with Hybrid Columnar Compression to deliver a highlycompact primary data warehouse footprint along with near-zero-initial-size data martsextracted from that small primary data warehouse. Such a setup can result insignificant savings in terms of storage footprint as well as enhanced performanceduring provisioning.In addition to integration with Oracle Enterprise Manager, it’s also important to havegranular insight into the storage itself in order to accurately tune and troubleshoot.Oracle ZFS Storage Appliance uniquely offers detailed drill-down, even to thepluggable database level, as a result of its co-engineering with the database. Theextended statistics available with the unique Oracle Intelligent Storage Protocoltechnology allows DBAs and storage administrators to resolve problems quicker withvisibility and correlations such as the help of AWR-like statistics made available on theappliance. Better visibility into the database-storage interaction replaces guessworkwith knowledge and confidence.C O N N E C T W I T H U S /oracle /oracle/oracle F O R M O R E I N F O R M A T I O N Contact: 1.800.ORACLE1 Oracle ZFS Storage Appliance offers multiprotocol support for Oracle Database. WhenOracle ZFS Storage Appliance is used in conjunction with Oracle Database 11g andhigher releases, the preferred protocol is Oracle’s Direct NFS Client —a high-performance NFS client feature built into Oracle Database. Direct NFS Client avoidsinefficiencies inherent in many OS kernel NFS implementations and achieves FC-likeperformance but with the simplified high availability and the infrastructure andmanagement cost characteristics of an enterprise Ethernet environment. For moreabout Direct NFS Client, please see the “Oracle Database 11g Direct NFS Client ”white pape.While Direct NFS Client is typically implemented over a 10 Gb Ethernet network, 40Gb InfiniBand is a connectivity option on Oracle ZFS Storage Appliance, making IPoIBwith Direct NFS Client a high-performance connectivity option. In addition, blockprotocols including iSCSI and FC are fully supported and widely used in cases whencustomers have a management preference for LUNs.Integration with Secondary Oracle Database StorageWorkloadsOne of the main benefits of using Oracle ZFS Storage Appliance for BIDW and otherprimary storage use cases is seamless integration with other Oracle Database storageworkloads. With built- in low-footprint snapshot and cloning data services, the primarydatabase can be quickly “copied” with little space and minimal overhead requirementsto meet development, test, or QA storage needs. Also, with built-in remote replicationplus best practices for Oracle Recovery Manager (Oracle RMAN) and Data Guard,Oracle ZFS Storage Appliance offers a number of options for both disaster recoveryand for backups integrated with development and test use cases that are fullycompatible with the primary database on Oracle ZFS Storage Appliance. Bestpractices for Oracle ZFS Storage Appliance are available for all of the followingsecondary workloads:» Backup using Oracle Database and Oracle RMAN» Disaster recovery using Data Guard» Provisioning development and test workloads from production, backup, or DRdatabases, including via the Snap Management Utility for Oracle Database featureof Oracle ZFS Storage Appliance (see the data sheet for more information)» Database consolidationConclusionAs engineered storage, Oracle ZFS Storage Appliance provides unique integrationswith Oracle Database to streamline management, provide superior efficiency, andaccelerate BIDW workloads in ways that are unobtainable by other vendors.Furthermore, Oracle ZFS Storage Appliance’s unique combination of a scalablestorage operating system, high-throughput system design, intelligent caching andflexible flash/disk/cloud configurations delivers the performance and availabilitycharacteristics needed for demanding mixed workload and BIDW environments.Oracle ZFS Storage Appliance is deployed by companies worldwide and proven inOracle’s own data centers, where Ora cle ZFS Storage Appliance is used for a varietyof database storage workloads. See the white paper, “Oracle ZFS Storage ApplianceC O N N E C T W I T H U S /oracle /oracle/oracle F O R M O R E I N F O R M A T I O N Contact: 1.800.ORACLE1 and Oracle IT: Use Cases and Benefits ” for more information.With compelling performance, unique Oracle co-engineering, attractive upfront pricing,and reduced operational costs due to simplified management, Oracle ZFS StorageAppliance is the ideal fit for your Oracle Database BIDW storage requirements.。
H C I E S t o r a g e知识点汇总统一存储LELE was finally revised on the morning of December 16, 2020HCIE-Storage知识点汇总统一存储系统概述SAN主要协议SCSI协议SCSI 指的是一个庞大的协议体系,到目前为止经历了SCSI-1/SCSI-2/SCSI-3 SCSI 协议定义了一套不同设备(磁盘、磁带、处理器、光设备,网络设备)等利用该框架进行信息交互的模型和必要指令集SCSI 的目标器 target和启动器 InitiatorFC协议基于FC光网络的传输协议,具有高速率,高效率,组网灵活FC的网络协议栈共分5层h,分别为FC-0,FC-1,FC-2,FC-3,FC-4FC-0:物理接口FC-1:数据链路层,编码/解码FC-2:网络层,封装/数据传递FC-3:公共服务,数据加密和压缩Fc-4:协议映射层光纤网的拓扑:点对点(2个设备)、光纤仲裁环(126个设备)、交换式光纤网(1600万)SAS协议SMP:SAS设备之间的信息SSP: SAS域中兼容SCSI命令STP: SAS域中兼容SATA命令iSCSI协议基于iSCSI协议的IP-SAN 是把用户的请求转换成SCSI代码,并将数据封装进IP包在以太网中进行传输iSCSI协议的优点:成本低、功能强、安全性高、带宽高、传输距离长以太网、TOE网卡、 iSCSI HBA卡FCoE 协议直接在增强型无损以太网基础设施上传输光纤信道信号的功能的协议。
FCoE把FC帧封装在以太网帧中,允许LAN和SAN业务流量在同一以太网中传送。
以太网可以容忍网络丢包,但FCoE协议不允许出现丢包IB协议IB 特性:高带宽、低延时、高可扩展性、支持QOS,支持RDMAPCI-E协议PCI-E总线是第三代高性能I/O总线,使用双向串行点对点传输,带宽高、功耗低、完善的错误检测和处理、有流控制,支持热插拔。
Linux服务器ZFS文件系统使用攻略ZFS(Zettabyte File System)作为一个全新的文件系统,全面抛弃传统File System + Volume Manager + Storage(文件系统+卷管理+存储)的架构,所有的存储设备是通过ZFS 池进行管理,只要把各种存储设备加入同一个ZFS 池,大家就可以轻松的在这个ZFS 池管理配置文件系统。
ZFS 包括一系列具有分层结构的存储元素,其中既有物理存储元素,又有逻辑存储元素。
所有这些元素都以有助于方便管理的方式相关联。
如下图,是ZFS文件系统与传统文件系统的对比。
图1 ZFS文件系统与传统文件系统的对比图一、为Linux服务器配置安装ZFS文件系统(1) 为rhel 配置EPEL repo如果既想获得RHEL 的高质量、高性能、高可靠性,又需要方便易用(关键是免费)的软件包更新功能,那么Fedora Project 推出的EPEL(Extra Packages for Enterprise Linux)正好适合你。
EPEL(/wiki/EPEL) 是由Fedora 社区打造,为RHEL 及衍生发行版如CentOS、Scientific Linux 等提供高质量软件包的项目。
装上了EPEL,就像在Fedora 上一样,可以通过yum install package-name,随意安装软件。
安装EPEL 非常简单:RHEL 6 系列使用:# rpm -Uvh/pub/epel/beta/6/i386/epel-release-6-1.noarch.rpmRHEL 5 系列使用:#rpm -Uvh/pub/epel/5/i386/epel-release-5-3.noarch.rpm(2)安装zfs-fuse模块# yum install zfs-fuse(3)从源代码安装如果要源代码安装首先安装如下模块:su -c yum install -y fuse-devel libattr-devel libaio-devel libacl-devel zlib-devel fuse-devel scons openssl-devel然后下载/releases/0.6.9/zfs-fuse-0.6.9.tar.bz2#/configure;make;make install加载zfs内核模块#modprobe zfs二、在Linux服务器上使用ZFS文件系统1了解zfs管理命令Zfs命令包括一组子命令主要如下:create 创建zfs文件系统destroy 摧毁一个ZFS文件系统snapshot 建立一个文件系统的快照rollback 从一个文件系统的快照中恢复clone 建立一个文件系统的克隆promote 从一个克隆创建一个文件系统upgrade 升级ZFS 文集系统list查看和询问数据集的信息allow 将用于执行ZFS 管理任务的细粒度权限委托给非特权用户unallow 将用于执行ZFS 管理任务的细粒度权限删除share 共享zfs文件系统unshared 取消共享zfs文件系统rename 重命名ZFS 快照mount 挂载zfs文件系统umount 卸载zfs文件系统set 可以设置或修改数据集的属性get 得到文件系统的一个专门属性zpool命令包括一组子命令如下:create 使用指定的实际设备建立存储池destroy摧毁一个ZFS存储池,但是不删除设备中数据add 在存储池中添加虚拟设备remove 在存储池中删除虚拟设备,但是不删除设备中数据list 显示所有存储池iostat查看存储池I/O状况status 查看存储池健康状况online把存储池状态设置为在线offline把存储池状态设置为离线clear 消除存储池设备错误计数attach 固定一个设备在存储池中detach 从存储池中分离设备replace 替换存储池中的设备scrub 校验存储池import 导入存储池export 导出存储池upgrade 升级存储池history 显示所有存储池操作命令get 找回和列出存储池的设备set 设置一个或者多个设备在一个存储池2 zfs使用实例:(1) 使用losetup 建立虛擬磁盘$ mkdir zfstest$ cd zfstest$ dd if=/dev/zero of=disk1.img bs=64M count=1 $ dd if=/dev/zero of=disk2.img bs=64M count=1 $ dd if=/dev/zero of=disk3.img bs=64M count=1 $ dd if=/dev/zero of=disk4.img bs=64M count=1 $ dd if=/dev/zero of=disk5.img bs=64M count=1 (2)建立简单存储池#zpool create -f zfstest1 /dev/loop0使用-f选项强行创建存储池和文件系统。
/bonwick/zh/category/ZFS ZFS介绍文件系统的工作归结为:当被请求读取块时,它应该返回上次写到此块的数据。
如果它做不到这一点(由于磁盘脱机或数据已经被损坏或篡改),那么它应该能检测并返回错误。
难以置信的是,大多数文件系统未能做到这一点。
它们依赖底层硬件来检测和报告错误。
如果磁盘返回坏数据,一般文件系统甚至无法检测到这一点。
即使我们可以假设所有磁盘都是完美的,数据也很容易在传输中受到损坏:控制器bug、DMA 奇偶校验错误等等。
您知道的只是数据离开磁盘片时是完好无缺的。
如果将数据当作一个包,则这就像UPS 所说的那样,“我们保证您的包裹在我们包装时没有损坏”。
这根本不是您寻求的保证。
过程中的损坏不仅仅是一个学术问题:甚至像糟糕的供电这样的小事都可能导致无提示数据损坏。
任何昂贵的存储阵列都无法解决这个问题。
I/O 路径仍然易受攻击,但它变得更长:不管阵列存在什么样的硬件和固件bug,在离开磁盘片之后,数据必须要能够存活。
如果您位于SAN 上,您将使用磁盘固件读盘器设计的网络。
上帝保佑您。
怎么办?一种选择是在每个磁盘块中存储校验和。
大多数现代磁盘驱动器可以使用比通常的512 字节略大的扇区来格式化,通常是520 或528。
这些额外的字节可用于存放块校验块。
但有效利用此校验和要比想象中困难:校验和的有效性极大地依赖于它的存储位置和它的评估时间。
在许多存储阵列中(请参阅Dell|EMC PowerVault paper 中的一个典型示例,当中有这些问题的完整描述),数据与其阵列内的校验和进行比较。
不幸的是,这没有多少用处。
它不会检测常见的固件bug,比如幻象写(上一次写操作,但实际上从未对磁盘执行过),因为数据和校验和存储为单元,所以它们是自我一致的(self-consistent),即使当磁盘返回坏数据时。
从阵列到主机的其他I/O 路径仍不受保护。
简而言之,这种块校验和提供了一种好的方法来确保阵列产品不会比它所包含的磁盘可靠性低,但也仅此而已。
storage的用法总结大全storage的用法总结大全精选3篇(一)1. 存储数据:storage可以用于存储各种类型的数据,例如文本、图片、视频等。
2. 数据备份:storage可以用于备份重要的数据,确保数据不会丢失。
3. 扩展存储空间:storage可以用作额外的存储空间,当本地设备存储空间不足时,可以将数据存储到外部存储设备中。
4. 云存储:storage可以用于将数据存储到云端,以便随时随地访问和共享数据。
5. 文件共享:storage可以用于共享文件,多个用户可以访问存储中的文件并进行编辑和更新。
6. 数据同步:storage可以用于将数据同步到不同设备之间,确保数据在多个设备上保持一致。
7. 数据恢复:storage可以用于恢复已删除或丢失的数据,通过备份的数据可以将数据恢复到原来的状态。
8. 数据加密:storage可以用于对敏感数据进行加密,确保数据的安全性和保密性。
9. 数据分析:storage可以用于存储大量的数据,以便进行数据分析和挖掘,帮助企业做出更好的决策。
10. 数据归档:storage可以用于归档不经常访问的数据,以节省存储空间和成本。
storage的用法总结大全精选3篇(二)- Storm是一个开源的实时大数据处理系统,用于处理实时数据流。
它可以与Hadoop 集成,提供高性能的实时数据处理能力。
- Storm可以用于实时分析和处理大规模数据流,如日志数据、传感器数据等。
它可以处理来自不同数据源的数据流,并将数据流分发到不同的处理单元进行处理。
- Storm使用一种称为拓扑(Topology)的方式来描述数据处理流程。
拓扑是由多个处理单元(称为Bolt)和连接它们的数据流(称为Spout)组成的。
- Spout可以从数据源中读取数据,并将数据流发射给Bolt进行处理。
Bolt可以对数据进行转换、过滤、聚合等操作,并将结果发射给下一个Bolt进行处理。
多个Bolt可以并行地执行不同的处理任务。
Oracle ZFS Storage ZS7-2 Getting Started GuideE93300-04November 2022Contents1 Oracle ZFS Storage ZS7-2controller2 Two power cords3 Slide rail kit4 Cable management arm5 One 6-meter Ethernet cable (black)6 Three 2.5-meter Ethernet cables (black, yellow, green)7 Up to eight (mid-rangemodel) or up to sixteen (high-end model) 3-meter or 6-meter active optical SAS cables (blue). Length andquantity per customer request,and sold separately.Front Panel Components1 RFID tag2 System status LED: Locator (white)3 System status LED: Service Action Required (amber)4 System status LED:Power/OK (green)5 Power button6 Fan service action required (amber)7 Power Supply (PS) service action required (amber)8 Over-temperature warning (amber)9 Service Processor (SP) OK (green)10 Do Not Service indicator 11 System disk 012 System disk 113 Filler panel 14 Filler panel 15 Filler panel 16 Filler panel 17 Filler panel 18 Filler panel 19 Filler panel 20 Filler panel21 Filler panel22 Filler panelBack Panel Components1 Power Supply Unit (PSU) 12 PSU 1 status indicators (topto bottom):•Attention (amber)•Power/OK (green)3 PSU 04 PSU 0 status indicators (topto bottom):•Attention (amber)•Power/OK (green)5 Fifth PCIe option (slot 1)6 External SAS-3 HBA (slot2)7 Slot 3:•High-end model: ExternalSAS-3 HBA•Mid-range model: Fillerpanel8 First PCIe option (slot 4)9 Third PCIe option (slot 5)10 Cluster interface card (slot 6)11 Serial management (SERMGT) RJ-45 serial port12 Service Processor (SP)NET MGT port13 Network (NET)100/1000BASE-T RJ-45 Gigabit Ethernet (GbE) port: NET 014 USB 3.0 port15 System status LEDs:•Locate (white) - left•Attention (amber) - right,top•Power/OK (green) - right,bottom16 Fourth PCIe option (slot 7)17 Second PCIe option (slot8)18 Slot 9:•High-end model: External SAS-3 HBA•Mid-range model: Filler panel19 External SAS-3 HBA (slot 10)20 Internal SAS-3 HBA (slot 11)Install the HardwareFor detailed information, see the Oracle ZFS Storage Appliance Installation Guide available from https:///en/storage.Connect System CablesFor detailed information, see the Oracle ZFS Storage Appliance Installation Guide available from https:///en/storage. If you have clustered controllers, connect the cables to both controllers. Do not connect power cables to the controllers or disk shelves until instructed to do so in "Power On and Configure the System." The following illustration shows an Oracle ZFS Storage ZS7-2 high-end model.Connect Clustered ControllersIf you have clustered controllers, connect the two controllers together. For detailed information, see the Oracle ZFS Storage Appliance Cabling Guide available from https:///en/storage.Connect Disk ShelvesThe following illustration shows clustered controllers connected to an Oracle Storage Drive Enclosure DE3-24P (left panel), and connected to four Oracle Storage Drive Enclosure DE3-24x disk shelves (right panel). For additional configuration options, see Oracle ZFS Storage Appliance Cabling Guide available from https:///en/storage. Do not connect power cables to disk shelves or controllers until instructed to do so in "Power On and Configure the System."The following procedure connects an AOC cable to the controller and to an Oracle Storage Drive Enclosure DE3-24P disk shelf. For additional information, see Oracle ZFS Storage Appliance Cabling Guide available from https:///en/ storage.1.Facing the controller HBA, position the cable's release tab to the right (left panel)for the controller end, and to the top for the disk shelf end (#1 in right panel).2.Insert the cable connectors into the appropriate ports, aligning the cable connectorlatches (#2 in right panel) with the port latch receiver holes (#3 in right panel).Ensure the latches engage with the receiver holes.Power On and Configure the SystemRequired information: Network host name and port of your serial connection, or network management port DHCP-assigned address. Connect disk shelf power cables, power on disk shelves, connect controller power cables, power on controllers, and configure the system as described in the Oracle ZFS Storage Appliance Installation Guide available from https:///en/storage.Perform Initial ConfigurationFor detailed information, see the Oracle ZFS Storage Appliance Installation Guide available from https:///en/storage, including properly configuring management interfaces and updating controller software.Product Documentation LibraryDocumentation for this product is available in the storage section of the Oracle Help Center, located at https:///en/storage.Access to Oracle SupportOracle customers that have purchased support have access to electronic support through My Oracle Support. For information, visit /pls/topic/ lookup?ctx=acc&id=info or visit /pls/topic/lookup?ctx=acc&id=trs if you are hearing impaired.FeedbackProvide feedback on this document at /goto/docfeedback.Oracle ZFS Storage ZS7-2 Getting Started GuideE93300-04Copyright © 2018, 2022, Oracle and/or its affiliates.This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and are protected by intellectual property laws. Except as expressly permitted in your license agreement or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license, transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means. Reverse engineering, disassembly, or decompilation of this software, unless required by law for interoperability, is prohibited.The information contained herein is subject to change without notice and is not warranted to be error-free. If you find any errors, please report them to us in writing.If this is software, software documentation, data (as defined in the Federal Acquisition Regulation), or related documentation that is delivered to the U.S. Government or anyone licensing it on behalf of the U.S. Government, then the following notice is applicable:U.S. GOVERNMENT END USERS: Oracle programs (including any operating system, integrated software, any programs embedded, installed, or activated on delivered hardware, and modifications of such programs) and Oracle computer documentation or other Oracle data delivered to or accessed by U.S. Government end users are "commercial computer software," "commercial computer software documentation," or "limited rights data" pursuant to the applicable Federal Acquisition Regulation and agency-specific supplemental regulations. As such, the use, reproduction, duplication, release, display, disclosure, modification, preparation of derivative works, and/or adaptation of i) Oracle programs (including any operating system, integrated software, any programs embedded, installed, or activated on delivered hardware, and modifications of such programs), ii) Oracle computer documentation and/or iii) other Oracle data, is subject to the rights and limitations specified in the license contained in the applicable contract. The terms governing the U.S. Government's use of Oracle cloud services are defined by the applicable contract for such services. No other rights are granted to the U.S. Government.This software or hardware is developed for general use in a variety of information management applications. It is not developed or intended for use in any inherently dangerous applications, including applications that may create a risk of personal injury. If you use this software or hardware in dangerous applications, then you shall be responsible to take all appropriate fail-safe, backup, redundancy, and other measures to ensure its safe use. Oracle Corporation and its affiliates disclaim any liability for any damages caused by use of this software or hardware in dangerous applications.Oracle®, Java, and MySQL are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners.Intel and Intel Inside are trademarks or registered trademarks of Intel Corporation. All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. AMD, Epyc, and the AMD logo are trademarks or registered trademarks of Advanced Micro Devices. UNIX is a registered trademark of The Open Group.This software or hardware and documentation may provide access to or information about content, products, and services from third parties. Oracle Corporation and its affiliates are not responsible for and expressly disclaim all warranties of any kind with respect to third-party content, products, and services unless otherwise set forth in an applicable agreement between you and Oracle. Oracle Corporation and its affiliates will not be responsible for any loss, costs, or damages incurred due to your access to or use of third-party content, products, or services, except as set forth in an applicable agreement between you and Oracle.。
ZFS文件系统和Sun x4500数据服务器在互联网行业的应用Xinfeng.liu@2008年6月简介在Web 2.0的时代,互联网上的数据呈现了爆炸性的增长。
大型互联网客户迫切需要一种大容量、廉价、节省空间的存储解决方案。
Sun Fire x4500服务器结合Solaris的ZFS文件系统很好地满足了这种需求。
Sun Fire X4500 服务器可以提供四路 x64 服务器的卓越性能,最多可在 4U 机架空间内提供 48 TB 最高密度存储。
该系统包含2 个双核 AMD Opteron 处理器,16GB内存,并内置了6个SATAII的磁盘控制器,48块热插拔的磁盘,以及4个千兆网口。
该系统可以提供高数据吞吐量,而成本只是传统解决方案的一半。
Solaris 10包含的ZFS文件系统是对传统文件系统的一次革命性的创新设计,ZFS解决了文件系统的完整性、安全性和可伸缩性以及管理困难等重要难题。
ZFS文件系统是世界上第一个128位的文件系统,其存储容量和文件数量几乎只受硬件的限制,非常适合互联网上的大数据量的应用。
而且ZFS文件系统的checksum和transactional 的操作方式实现了端对端的数据安全,减少了为保证数据的高可靠性对昂贵硬件的依赖,大大降低了存储成本。
另外,ZFS文件系统的管理非常简便,采用了存储池的管理,并包含了传统的卷管理器的功能,但无需传统卷管理器的复杂的命令操作。
由于ZFS的独特设计,ZFS文件系统无需mount,无需fsck,无需journaling,大大地简化了日常维护。
ZFS文件系统已包含在Solaris 10或opensolaris中,无需另外下载并且是免费的和开源的。
本文主要介绍ZFS文件系统和Sun x4500服务器作为互联网上文件服务器的典型应用和优化。
其内容包括:●ZFS文件系统的特点和调优●ZFS文件系统的部署规划建议●Sun x4500作为文件服务器的系统调优ZFS文件系统的特点和调优ZFS文件系统在设计上与传统的文件系统有很大的不同。
ZFS是什么?使⽤ZFS的理由及特性介绍ZFS 的历史Z ⽂件系统(Z File System)(ZFS)是由 Matthew Ahrens 和 Jeff Bonwick 在 2001 年开发的。
ZFS 是作为太阳微系统(Sun MicroSystem) 公司的 OpenSolaris 的下⼀代⽂件系统⽽设计的。
在 2008 年,ZFS 被移植到了 FreeBSD 。
同⼀年,⼀个移植ZFS 到 Linux 的项⽬也启动了。
然⽽,由于 ZFS 是通⽤开发和发布许可证 (Common Development and Distribution License)(CDDL)许可的,它和 GNU 通⽤公共许可证不兼容,因此不能将它迁移到 Linux 内核中。
为了解决这个问题,绝⼤多数Linux 发⾏版提供了⼀些⽅法来安装 ZFS。
在甲⾻⽂公司收购太阳微系统公司之后不久,OpenSolaris 就闭源了,这使得 ZFS 的之后的开发也变成闭源的了。
许多 ZFS 开发者对这件事情⾮常不满。
三分之⼆的 ZFS 核⼼开发者,包括 Ahrens 和 Bonwick,因为这个决定⽽离开了甲⾻⽂公司。
他们加⼊了其它公司,并于 2013 年 9 ⽉创⽴了 OpenZFS 这⼀项⽬。
该项⽬引领着 ZFS 的开源开发。
让我们回到上⾯提到的许可证问题上。
既然 OpenZFS 项⽬已经和 Oracle 公司分离开了,有⼈可能好奇他们为什么不使⽤和GPL 兼容的许可证,这样就可以把它加⼊到 Linux 内核中了。
根据 OpenZFS 官⽹的介绍,更改许可证需要联系所有为当前OpenZFS 实现贡献过代码的⼈(包括初始的公共 ZFS 代码以及 OpenSolaris 代码),并得到他们的许可才⾏。
这⼏乎是不可能的(因为⼀些贡献者可能已经去世了或者很难找到),因此他们决定保留原来的许可证。
ZFS 是什么,它有什么特性?正如前⾯所说过的,ZFS 是⼀个先进的⽂件系统。
<Insert Picture Here>更灵活,更易用-----Oracle ZFS St 更灵活 更易用 O l Storage 统一存储介绍存储需求持续大幅度成长P PBFile Based: 79.3% CAGRSource: IDCBlock based: 31% CAGR2存储需求持续大幅度成长Greater Demand For Storage Capacity And Performance2009 New Digital Data2020 New Digital Data44X Growth800 Exabytes 35,000 Exabytes•© 2010 Oracle Corporation – Proprietary and Confidential3定义存储效率(Storage Efficiency) ( g y)• 使用最低成本来进行数字数据的存放 保护及管理 使用最低成本来进行数字数据的存放、保护及管理 • 可降低数据中心的电力 散热及空间成本 可降低数据中心的电力、散热及空间成本4存储已经成为数据中心 最大的成本提高存储效率数据中心最迫切的需求• 数据存储的效率必须提高• 根据调查用户将其一份资料重复存放超过15份 • 政府法规对数据保留需求持续上升• 数据重复删除及数据压缩是关键需求• 有50%的IDC响应目前有使用数据重复删除的需求• 电力、散热及机架空间效率是不可缺少的评估要点 • 存储集中化是客户IT计划里面排名前10名的重点项目5Oracle ZFS Storage 统一存储系统重新定义统一存储系统(Unified Storage)• 同时提供文件(File)及数据块(block)服务 ( ) ( )• 只要买一套硬件,一个价格,一次提供10种数据通讯服务• 内置先进的数据服务• 集群(Clustering),远程数据复制(Replication), 快照( Snapshot),数 据复制(Cone),数据重复删除(Deduplication),数据压缩 (Compression),自动精简配置(thin provisioning),闪存(Flash), (Compression) 自动精简配置(thin provisioning) 闪存(Flash) 在 线实时分析功能( Analytics),病毒扫描(Virus scan)• 可选择不同的主机接口(interconnect)• 可同一台机器安装以太网接口(Ethernet), 光纤 (Fibre channel)及 Infiniband• 可选择的应用程序• Oracle Solaris或Linux, Windows, virtualization•66Oracle ZFS Storage g全新第二代统一存储系统•标准功能 •内含所有主流通讯协议 •7120 •7320 •内含先进的数据服务功能 •入门型l • • 入门型l •7420 •单控制器机型 支持操作系统及应用程序 •单控制器机型 •Oracle Solaris • Oracle Enterprise Linux •高扩充能力机型 •支持集群功能 •Oracle VM • VMware • Windows •Oracle数据库、 Oracle Middleware及Oracle应用程序等 支持超过50种以上的商业应用程序 •7720 • 新的产品特点 •高密度存储机型 •密度及容量:可集中化存储容量最大可达1PB •支持集群功能 •闪存容量倍增: 最大可达4TB读取闪存及432GB写入闪存 •更多的运算能力: 每个控制器最大可达32个核,效能比前一代提升50%•77完整的存储功能下列功能皆包含在ZFS Storage上-不需额外付费购买 Storage上 不需额外付费购买数据通讯协义• Fibre Channel • iSCSI • Infiniband over IP/RDMA • iSER • SRP • NFS v3 and v4 • CIFS • HTTPNew! • WebDAV数据服务数据管理• FTP • ZFS NDMP v4• Browser and CLI Interface • Single, Double & Triple Parity RAID (RAIDZ, Z2, Z3) • Management Dashboard • Mi Mirroring & Triple Mirroring i T i l Mi i • Hardware/component view • Hybrid Storage Pool • Role-based Access Control • End-to-End Data Integrity • Remote Replication • Phone Home • Snapshots and Clones • Event and Threshold based • Quota(s) Alerting • In-line Dedup • Dtrace Analytics • Compression • Scripting • Thin Provisioning • Antivirus via ICAP Protocol • Workflow Automation • Online Data Migration New!Advanced Networking • • Clustering g •DFS Standalone New! Namespace •Source Aware Routing•Oracle Internal and Confidential8内置先进的存储技术提供更高的使用效率进一步让存储系统更有效率 • 内置重复数据删除技术(in-line, Deduplication) • 降低整体数据存放的存储空间 • 内置数据压缩(Compression) • 整合重复数据删除技术让空间使用更加有效率 • 内置存储自动精简配置(Thin-provisioning) • 能够有效的满足客户端容量需求并优化存储系统的空间使用率 • 内置混合存储池架构(Hybrid Storage Pools, HSPs) • 整合内存,闪存及大容量硬盘科技降低电力及机架空间的需求 • 内置多样先进的数据保护功能(D 内置多样先进的数据保护功能(Data P Protection capabilities) i bili i ) • 快照+数据远程复制,允许3块数据盘故障的RAID-Z3等先进数据 保护功能9Oracle先进的重复数据删除技术让使用存储 空间上更有效率• 实时重复数据删除:当数据产生时就进行重复数据删除操作 实时重复数据删除 当数据产生时就进行重复数据删除操作• 当友商存储系统使用重复数据删除操作时,必须是采用列表(Schedule)方式并于背景模 式(POST)下运作,会大幅降低存储效能 • Sun ZFS存储系统因为有强大的运算处理器及内存可以实时进行重复数据删除功能• Oracle重复数据删除使用256-bit的检查更严谨• ZFS存储系统的重复数据技术采用256位同位检查(256-bit checksum)比起竞争友商的 重复数据技术只采用16位同位检查(16 bit 重复数据技术只采用16位同位检查(16-bit checksum)更加严谨• Oracle重复数据删除可以于更大的文件系统上运作• ZFS存储系统的数据重复删除技术可以在单一576TB的文件系统上运作,比竞争友商的 存储系统的数据重复删除技术只能在单一16TB的文件系统上运作更有效率• Oracle采用数据块(Block-level)进行重复数据比对• 数据块比文件的重复数据删除技术对于虚拟主机的使用环境上更有效率10内置ZFS数据压缩功能驱动更高的效率及效能• 内置ZFS压缩功能• 减少整体磁盘使用空间• 客户实际使用各种不同非结构化数据的应用程序验证,确实可以大幅度减少整体磁盘使用空间 • 一般情况下有2倍压缩率 (或50%空间减少)• 减少整体数据量,提高有效输出带宽(Throughput)• 不仅是节省存储空间,还提升了存储系统效能 • 数据压缩功能启动时可以让数据快速读写,减少存取(I/O)频率• 可以与数据重复删除功能一起使用有加倍的效果• 提供无与伦比的硬件处理能力11•玛拉基.麦凯布 玛拉基 麦凯布 •(Malachi McCabe) •经理 Signature Styles • 信息管理部门 •Signature Styles Signature “我们看到于ZFS统一存储系统可以对公司内的非结构化数据提供多达 我们看到于ZFS统 存储系统可以对公司内的非结构化数据提供多达 73%的数据压缩比例”12ZFS混合存储池于存储集中化的环境下提供效能且帮助客户省钱• ZFS混合存储池可以智能判断数据使用状况及自动 在内存,闪存及磁盘之间迁移 • 不断优化存储系统性能和效率–优化 $/GB/s及 $/IOP 效能 • 管理单一混合存储池是非常简单的•内存 •(DRAM) •写入用 闪存(ZIL)•读取闪存 (L2ARC)“我最感兴趣的是,混合存储池提供不亚于内存等级 的速度(或称接近内存等级的效能)来存取Oracle 能 数据库内的数据。
…混合存储池架构是我们决定要 使用通用存储设备的重要原因之 使用通用存储设备的重要原因之一”• 戴维.罗比拉德– UNIX团队领导及Oracle数据库管理员, Notarius•可扩充的大 容量SAS磁盘 驱动器13ZFS混合存储池原理概述自动调整性能瓶颈区域旧模式存储池结构 模式存储 结构新的存储池结构混合存储池 (ZFS)• ARC – ZFS统一存储系统缓存; • L2ARC – 可采用读固态硬盘,作为ZFS统一 存储的二级缓存;根据数据访问频繁度,动 存储的二级缓存 根据数据访问频繁度 动 态加载相关数据,提高数据读取性能; • ZIL – 可采用写固态硬盘,作为ZFS文件的系 统Log存放空间,提高数据写性能; • Disk Storage Pool – 采用SAS磁盘,提高大 容量存储空间。
14在电力、机架空间及散热效率上拥有划时代 的结果100块企业级高转速硬盘 ZFS混合存储池架构•2 SSDs•15块大容量硬盘容量: 30TB 效能: 30K IOPS 所需电力: 1,750W容量: 30TB 效能: 40K IOPS 所需电力: 392W•只需要¼的电力,1/6的机架空间 只需要¼的电力,1/6的机架空间1515时间效率(Time Efficiency)413小时•...每年会花这么久的时间来彻底解决一个文件/应用程序 的性能问题…— EMC Study16使用效能及时分析工具(Dtrace)来提升时效• 快速且简易的收集及管理效能热 点及瓶颈 • 了解如何配置和应用程序的变化会影响 到存储系统 • 可检测所有的存储设备(内存,固态硬盘, 磁盘驱动器)不需猜测 •根据最近一份公正第三方的研究报告: ZFS通用存储系统与NetApp存储产品同 pp 时比较时, ZFS通用存储系统可以减少管理的复杂度达30%并且提升存储管理人 员生产力达35%17•戴维.罗比拉德 (David Roillard) Unix Team Leader & Oracle DBA, Notarius DBA “以我11年管理存储系统的经验,我从来没有从其他存储设备内部得知有 关于像存储系统 IOPS这么详细的信息 我之前曾经使用过市场上主流 IOPS这么详细的信息。