陈齐彦《容器打造持续交付和高可用架构的原力》pdf
- 格式:pdf
- 大小:1018.41 KB
- 文档页数:26
容器技术的高可用和容错机制近年来,容器技术的兴起让应用部署和管理变得更加便捷高效。
然而,随着企业对容器技术的广泛应用,对于容器的高可用性和容错能力也提出了更高的要求。
在容器技术的发展过程中,高可用和容错机制成为了关注的焦点之一。
一、高可用性的重要性高可用性是指系统或服务在面对故障或其他异常情况时仍能提供稳定的运行能力。
对于容器技术而言,高可用性是保证业务连续稳定运行的基础。
在容器应用中,发生故障可能导致整个应用的不可用,因此高可用性的保证显得尤为重要。
二、容器集群的高可用性容器技术的核心是容器编排工具,如Kubernetes等。
通过容器编排工具,用户可以创建一个由多个容器组成的集群,保证应用的高可用性。
在容器集群中,可以将应用的多个实例部署在不同的节点上,当某个节点发生故障时,其他节点可以接替其工作,保证应用的连续可用。
三、容器的自动重启为了提高容器的高可用性,常见的做法是设置容器的自动重启。
当容器发生故障或崩溃时,容器编排工具可以自动监测并重新启动容器,确保应用的持续运行。
这种方式可以大大减少人工干预的需求,提高了容器的可用性。
四、容器镜像的备份容器镜像是容器技术的核心组件之一,存储了应用的代码和依赖库等信息。
为了应对容器镜像的损坏或丢失,需要进行备份。
容器编排工具可以定期对容器镜像进行备份,并存放在云存储等可靠的地方。
当容器镜像损坏时,可以通过备份恢复镜像,提高容器的容错能力。
五、容器的故障迁移容器的故障迁移是容器技术中的常见机制,用于应对容器所在节点的故障。
当一个节点发生故障时,容器编排工具可以自动将该节点上的容器迁移至其他可用节点上,保证应用的可用性。
这种故障迁移机制可以将应用的停机时间减少到最低限度,提高了容器的容错性。
六、容器的定期健康检查为了及时发现容器的故障情况,容器编排工具会定期对容器进行健康检查。
通过检查容器的运行状态、资源消耗和网络连接等指标,可以及时发现异常并进行处理。
这种定期健康检查机制能够提前预警并解决潜在的容器故障问题,保证了容器的高可用性。
Integrated Resource Management for Cluster-based Internet Services Kai Shen Hong Tang Tao Yang Lingkun Chu kshen@ htang@ tyang@ lkchu@Dept.of Computer Science,University of California at Santa Barbara,CA93106Ask Jeeves/Teoma Technologies,Piscataway,NJ08854AbstractClient request rates for Internet services tend to be bursty and thus it is important to maintain efficient re-source utilization under a wide range of load work service clients typically seek services interactively and maintaining reasonable response time is often imperative for such services.In addition,pro-viding differentiated service qualities and resource al-location to multiple service classes can also be desir-able at times.This paper presents an integrated re-source management framework(part of Neptune sys-tem[34])that providesflexible service quality speci-fication,efficient resource utilization,and service dif-ferentiation for cluster-based services.This framework introduces the metric of quality-aware service yield to combine the overall system efficiency and individual ser-vice response time in oneflexible model.Resources are managed through a two-level request distribution and scheduling scheme.At the cluster level,a fully decen-tralized request distribution architecture is employed to achieve high scalability and availability.Inside each service node,an adaptive scheduling policy achieves ef-ficient resource utilization under a wide range of load conditions.Our trace-driven evaluations show that the proposed techniques can efficiently utilize system re-sources under quality constraints and provide service paring with a previously proposed dynamic server partitioning approach,the evaluations also show that Neptune responds more promptly to de-mand spikes and behaves more smoothly during server failures.1IntroductionPrevious studies show that the client request rates for Internet services tend to be bursty andfluctuate dra-matically[1,12,13].For example,the daily peak-to-average load ratio at Internet search service Ask Jeeves ()is typically3:1and it can be much higher and unpredictable in the presence of extraordi-nary events.Over-provisioning system resources for a service site to accommodate the potential peak will not be cost-effective.As a consequence,it is desirable to achieve efficient resource utilization for those services under a wide range of load conditions.Network clients typically seek services interactively and maintaining reasonable response times is imperative.In addition,providing differentiated service qualities and resource allocation to multiple service classes can also be desirable at times,especially when the system is reaching its capacity limit and cannot provide interactive responses to all the requests.Quality of service(QoS) support and service differentiation have been studied ex-tensively in network packet switching with respect to packet delay and connection bandwidth[9,14,25,36]. It is equally important to extend network-level QoS sup-port to endpoint systems where service fulfillment and content generation take place.Those issues are es-pecially critical for cluster-based network services in which contents are dynamically generated and aggre-gated[1,18,20,32,34].This paper presents the design and implementation of an integrated resource management framework for cluster-based services.This framework is part of Nep-tune system:a cluster-based software infrastructure for aggregating and replicating partitionable network ser-vices[33,34].Neptune has been successfully de-ployed at Internet search engine Ask Jeeves[1]sinceDecember2001.Although cluster-based network ser-vices have been widely deployed,we have seen lim-ited research in the literature on comprehensive resource management with service differentiation support.Re-cent studies on endpoint resource management and QoS support have been mostly focused on single-host sys-tems[3,4,7,8,10,26,38]or clustered systems serving static HTTP content[5,31].In comparison,Neptune is intended for clustered services with dynamic service fulfillment or content generation.The work presented in this paper addresses the inadequacy of the previous studies and complements them in the following three as-pects.Flexible resource management objectives.Most pre-vious studies have been using a monolithic metric to measure resource utilization and define QoS constraints. Commonly used ones include system throughput,mean response time,mean stretch factor[41],or the tail dis-tribution of the response time[27].Neptune introduces a unified metric that links the overall system efficiency with individual service response time.To be more spe-cific,we consider the fulfillment of a service request produces certain quality-aware service yield depending on the response time,which can be linked to the amount of economical benefit or social reach resulted from serv-ing this request.The overall goal of the system is to maximize the aggregate service yield resulted from all requests.As an additional goal,Neptune supports ser-vice differentiation for multiple service classes through two means:1)service classes can acquire differentiated service support by being configured to produce different service yield;2)each service class can also be guaran-teed to receive a certain proportion of system resources, if so requested.Fully decentralized clustering architecture.Scalabil-ity and availability are always overriding concerns for large-scale cluster-based services.Several prior studies have been relying on centralized components to manage resources for a cluster of replicated servers[5,12,31, 41].In contrast,our framework employs a functionally symmetrical architecture that does not rely on any cen-tralized components.Such a design not only eliminates potential single point of failure in the system,it is also crucial to ensuring a smooth and prompt response to de-mand spikes and server failures.Efficient resource utilization under quality con-straints.Neptune achieves efficient resource utilization through a two-level request distribution and schedul-ing scheme.At the cluster level,requests for each ser-vice class are evenly distributed to all replicated service nodes without explicit partitioning.Inside each service node,an adaptive scheduling policy adjusts to the run-time load condition and seeks high aggregate service yield at a wide range of load levels.When desired,the service scheduler also provides proportional resource al-location guarantee for specified service classes.The rest of this paper is organized as follows.Sec-tion2illustrates a target architecture for this work and then describes the multi-fold objective of our resource management framework.Section3presents Neptune's two-level request distribution and scheduling architec-ture.Section4illustrates the service scheduling inside each service node.Section5presents the system imple-mentation and experimental evaluations based on traces and service components from a commercial search en-gine.Section6discusses related work and Section7 concludes the paper.2Targeted Architecture and Resource Man-agement ObjectiveIn this section,wefirst illustrate the targeted system ar-chitecture of this work.Then we introduce the concepts of quality-aware service yield and service yield func-tions.Through these concepts,service providers can express a variety of quality constraints based on the ser-vice response time.Furthermore,using service yield functions and resource allocation guarantees,our frame-work allows service providers to determine the desired level of service differentiation among multiple service classes.2.1Targeted ArchitectureOur framework targets cluster-based network services accessible to many users through an intranet or the In-ternet.Inside those clusters,services are usually parti-tioned,replicated,aggregated,and then delivered to ex-ternal clients through protocol gateways.Figure1uses a prototype search engine to illustrate our targeted system architecture[1,2].In this example,the service cluster delivers search services to consumers and business part-ners through Web servers and XML gateways.Inside the cluster,the main search tasks are performed on a setof index servers and document servers,both partitioned and replicated.Each search queryfirst arrives at one of the protocol gateways.Then some index servers are contacted to retrieve the identifications of Web pages re-lated to the search query.Subsequently some document servers are mobilized to retrieve a short description of these pages and thefinal results are returned through the original protocol gateway.The resource management work in this study is focused on resources and quality constraints inside the service cluster.Issues related to wide-area network latency or bandwidth is out of the scope of this paper.Figure1:A targeted system architecture:search engine. Notice that a complicated service application can in-volve multiple tiers of clustered internal services.In such a case,a service cluster is divided into multiple tiers and each tier may request services provided by other tiers.While Neptune supports multi-tier services (e.g.Ask Jeeves search),this paper focuses on the re-source management problem within a single-tier cluster.2.2Quality-aware Resource UtilizationMost previous studies have been using a monolithic metric such as system throughput,mean response time, mean stretch factor[41],or the tail distribution of the response time[27]to measure the efficiency of sys-tem resource management.We use a more compre-hensive metric by conceiving that the fulfillment of a service request provides certain yield depending the re-sponse time.This yield,we call quality-aware service yield,can be linked to the amount of economical bene-fit or social reach resulted from serving this request in a timely fashion.Both goals of provisioning QoS and ef-ficient resource utilization can be naturally combined as producing high aggregate yield.Furthermore,Neptuneconsiders the service yield resulted from serving eachrequest to be a function of the service response time.The service yield function is normally determined byservice providers to give themflexibility in expressingdesired service qualities.Let,,,be the re-sponse times of the service accesses completed in anoperation period.Let represent the service yield function for the th service access.The goal of our sys-tem is to maximize the aggregate yield,i.e.maximize(1)In general,the service yield function can be any monotonically non-increasing function that returns non-negative numbers with non-negative inputs.We give a few examples to illustrate how service providers can use yield functions to express desired service qualities.For instance,the system with the yield function throughput depicted in Figure2(A)is intended to achieve high sys-tem throughput with a deadline.In other words,the goal of such a system is to complete as many service accesses as possible with the response time.Simi-larly,the system with the yield function resptime in Fig-ure2(B)is designed to achieve low mean response time. Note that the traditional concept of mean response time does not count dropped requests.resptime differs from that concept by considering dropped requests as if they are completed in.We notice that throughput does not care about the exact response time of each service access as long as it is com-pleted within the deadline.In contrast,resptime always reports higher yield for accesses completed faster.As a hybrid version of these two,hybrid in Figure2(C)pro-duces full yield when the response time is within a pre-deadline,and the yield decreases linearly thereafter. The yieldfinally declines to a drop penalty when the response time reaches the deadline.This corresponds to the real world scenario that users are generally com-fortable as long as a service request is completed in. They get more or less annoyed when the service takes longer and they most likely abandon the service after waiting for.represents the full yield resulted from a prompt response and the drop penalty represents the loss when the service is not completed within thefi-nal deadline.Figure2gives the illustration of thesethree functions.We want to point out that throughput is a special case of hybrid when;and resptime is also a special case of hybrid when and.In certain sense,our definition of service yield is sim-ilar to the value in value-based real-time database sys-tems[22,23].One major distinction is that real-time systems are usually sized to handle transient heavy load[22].For Internet services,however,the client re-quest rates tend to be bursty and over-provisioning sys-tem resources for a service site to accommodate the po-tential peak will not be cost-effective[1,12,13].More detailed discussions on scheduling schemes to achieve high aggregate service yield are given in Section4.2.2.3Service DifferentiationService differentiation is another goal of our multi-fold resource management objective.Service differentiation is based on the concept of service classes.A service class is defined as a category of service accesses that obtain the same level of service support.On the other hand,service accesses belonging to different service classes may receive differentiated QoS support.Service classes can be defined based on client identities.For instance,a special group of clients may be configured to receive preferential service support or a guaranteed share of system resources.Service classes can also be defined based on service types or data partitions.For example,a order placement transaction is typically con-sidered more important than a catalog-browsing request. Our framework provides differentiated services to dif-ferent service classes on two fronts.First,service classes can acquire differentiated service support by specifying different yield functions.For instance,serv-ing a VIP-class client can be configured to produce higher service yield than serving a regular client.Sec-ondly,each service class can be guaranteed to receive a certain portion of system resources.Most previous ser-vice differentiation studies have focused on one of the above two means of QoS support[8,24,29,39].We be-lieve a combination of them provide two benefits when system is overloaded:1)the resource allocation is bi-ased toward high-yield classes for efficient resource uti-lization;2)a certain portion of system resources can be guaranteed for each service class,if needed.The sec-ond benefit is crucial to preventing starvation for low-priority service classes.3Two-level Request Distribution and Schedul-ingIn our framework,each external service request enters the service cluster through one of the gateways and it is classified into one of the service classes according to rules specified by service providers.The gateway node then accesses one or more(in the case of service aggre-gation)internal services to fulfill the request.Inside the service cluster,each service can be made available at multiple nodes through replication.In this section,we discuss the cluster-level request distribution on a group of replicated servers for a single service.The dynamic partitioning approach proposed in a pre-vious study adaptively partitions all replicas for each service into several groups and each group is assigned to handle requests from one service class[41].We be-lieve such a scheme has a number of drawbacks.First,a cluster-wide scheduler is required to make server par-titioning decisions,which is not only a single-point of failure,but also a potential performance bottleneck. Secondly,cluster-wide server groups cannot be repar-titioned very frequently,which makes it difficult to re-spond promptly to changing resource demand.In order to address these problems,Neptune does not explicitly partition server groups.Instead,we use a symmetri-cal and decentralized two-level request distribution and scheduling architecture illustrated in Figure3.Figure3:Two-level request distribution and scheduling. Each service node in this architecture can process re-quests from all service classes.The resource manage-ment decision is essentially made at two levels.First,Figure2:Illustration of service yield functions.each service request is directed to one of the replicated service node through a cluster-level request distribution. Upon arriving at the service node,it is then subject to a node-level service scheduling.At the cluster level,Nep-tune employs a class-aware load balancing scheme to evenly distribute requests for each class to all servers. Our load balancing scheme uses a random polling policy that discards slow-responding polls.Under this policy, whenever a client is about to seek a service for a partic-ular service class,it polls a certain number of randomly selected service nodes to obtain the load information. Then it directs the service request to the node with the smallest number of active and queued requests.Polls not responded within a deadline are discarded.This strategy also helps excluding faulty nodes from request distribution.We use a poll size of3and a polling dead-line of10ms in our system.Our recent study shows that such a policy is scalable and well suited for services of a large spectrum of granularities[33].Inside each service node,our approach must also deal with the resource al-location across multiple service classes.This is handled by a node-level class-aware scheduling scheme,which will be discussed in Section4.Note that the node-level class-aware scheduling is not necessary for the server partitioning approach because every node is configured to serve a single service class under that approach.An Alternative Approach for Comparison.For the purpose of comparison,we also design a request distri-bution scheme based on server partitioning[41].Server partitioning is adjusted periodically atfixed intervals. This scheme uses the past resource usage to predict the future resource demand and makes different partitioning decisions during system under-load and overload situa-tions.When the aggregate demand does not exceed the total system resources,every service class acquires their demanded resource allocation.The remain-ing resources will be allocated to all classes pro-portional to their demand.When the system is overloaded,in thefirst round we allocate to each class its resource demand or its resource allocation guarantee,whichever is smaller.Then the remaining resources are allo-cated to all classes under a priority order.The pri-ority order is sorted by the full yield divided by the mean resource consumption for each class1. Fractional server allocations are allowed in this scheme. All servers are partitioned into two pools,a dedicated pool and a shared pool.A service class with2.4server allocation,for instance,will get two servers from the dedicated pool and acquire0.4server allocation from the shared pool through sharing with other classes with fractional allocations.The length of the adjustment interval should be cho-sen carefully so that it is not too small to avoid exces-sive repartitioning overhead and maintain system stabil-ity,nor is it too large to promptly respond to demand changes.We choose the interval to be10seconds in this paper.Within each allocation interval,service re-quests are randomly directed to one of the servers allo-cated to the corresponding service class according to the load balancing policy[33].4Node-level Service SchedulingNeptune employs a multi-queue(one per service class)scheduler inside each node.Whenever a service requestarrives,it enters the appropriate queue for the serviceclass it belongs to.When resources become available,the scheduler dequeues a request from one of the queuesfor service.Figure4illustrates such a runtime environ-ment of a service node.Class 1Class 2... ...Class NFigure4:Runtime environment of a service node. For a service node hosting service classes:,each class is configured with a ser-vice yield function and optionally a minimum systemresource share guarantee,which is expressed as a per-centage of total system resources().The goal of the scheduling scheme is to provide the guaran-teed system resources for all service classes and sched-ule the remaining resources to achieve high aggregateservice yield.Figure5illustrates the framework of ourservice scheduling algorithm at each scheduling point.In the rest of this section,we will discuss two aspects ofthe scheduling algorithm:1)maintaining resource allo-cation guarantee;and2)achieving high aggregate ser-vice yield.4.1Estimating Resource Consumption for Alloca-tion GuaranteeIn order to maintain resource allocation guarantee,weneed to estimate resource consumption for each serviceclass at each scheduling time.This estimation should bebiased toward recent usage to stabilize quickly when theactual resource consumption jumps from one level to an-other.It should not be too shortsighted either in order toavoid oscillations or over-reactions to short-term spikes.Among many possible functions that exhibit those prop-erties,we define the resource consumption for classat time to be the weighted summation of the resourceusage for all class requests completed no later than .The weight is chosen to decrease exponentially with regard to the elapsed time since the request completion. For each request,let be its completion time and1.Drop from each queue head those requests that arelikely to generate zero or very small yield accord-ing to the request arrival time,expected servicetime and the yield function.2.Search for the service classes with non-empty re-quest queues that have an estimated resource con-sumption of less than the guaranteed share.(Sec-tion4.1)(a)If found,schedule the one with the largestgap between the resource consumption andthe guaranteed share.(b)Otherwise,schedule a queued request thatis likely to produce high aggregate serviceyield.(Section4.2)Figure5:The node-level service scheduling algorithm.be its actual resource usage(we will discuss how to measure it in the end of this sub-section),which is known after its completion.Equation2definesto be the resource consumption for class at time. Note that the time in all the following equations is de-nominated in second s.and(2) Another reason for which we choose this function is that it can be incrementally calculated without maintaining the entire service scheduling history.If we adjustat the completion of every request and let be the previ-ous calculation time,the resource consumption at time can be calculated incrementally through Equation3.(3) The selection of should be careful to maintain the smooth and stable reaction for both short-term spikes and long-term consumption changes.In this paper we empirically choose to be0.95.Since we use secondas the unit of time in those equations,this means a ser-vice request completed one second ago carries95%theweight of a service request completed right now.With the definition of,the proportional re-source consumption of class can be represented byPriority(the smaller the higher)EDFYIDGreedyAdaptivethe scheduling time.For the response time,we use an exponentially-weighted moving average of the re-sponse time of past requests belonging to the same service class.Resource consumption measurement is application-specific as we have explained in the previ-ous sub-section.In our current implementation,such a prediction is based on an exponentially-weighted mov-ing average of the CPU consumptions of past requests belonging to the same service class.Such an approx-imation does not affect the applicability of the pro-posed scheduling policies as our evaluation in Section5 demonstrates.5System Implementation and Experimental EvaluationsNeptune has been implemented on a Linux cluster.In addition to the resource management framework de-scribed in this paper,Neptune provides load balancing and service replication with fail-over support for cluster-based services[33,34].Application developers can eas-ily deploy services through specifying a set of RPC-like access methods for each service and clients can access them through a simple programming API.Nep-tune employs a symmetrical architecture in construct-ing the service infrastructure.Any node can elect to provide services and seek services from other nodes in-side the service cluster.Each external service request is assigned a service class ID upon arriving at any of the gateways.Those requests are directed to one of the replicated service nodes according to the class-aware load balancing scheme.Each server node maintains multiple request queues(one per service class)and a thread pool.To process each service request,a thread is dispatched to invoke the application service module through dynamically-linked libraries.The size of the thread pool is chosen to strike the balance between con-currency and efficiency depending on the application characteristics.The aggregate services are exported to external clients through protocol gateways.Neptune has been subsequently ported into Solaris platform.An earlier version of this system has been successfully de-ployed in Ask Jeeves search since December2001[1]. Thefirst goal of our evaluation is to examine the system performance of our service scheduling schemes over a wide range of load conditions.Secondly,we will study the performance and scalability of our cluster-level re-quest distribution scheme.Our third goal is to investi-gate the system behavior in terms of service differenti-ation during demand spikes and server failures.All the evaluations were conducted on a rack-mounted Linux cluster with30dual400MHz Pentium II nodes,each of which contains either512MB or1GB memory.Each node runs Linux2.2.15and has two100Mb/s Ethernet interfaces.The cluster is connected by a Lucent P550 Ethernet switch with22Gb/s backplane bandwidth. 5.1Evaluation WorkloadsOur evaluation studies are based on two service work-loads.Thefirst service is a Differentiated Search ser-vice based on an index search component from Ask Jeeves search.This service takes in a group of encoded query words;checks an mmap-ed index database;and returns a list of URLs that are relevant to input query words.The index database size is around2.5GB at each node and it cannot completelyfit in memory.The mean service time for this service is around250ms in our testbed when each request is served in a dedicated environment.Differentiated Search distinguishes three classes of clients,representing Gold,Silver and Bronze member-ships.We let the request composition for these three classes be10%,30%,60%respectively.The yield func-tions of these service classes can be one of the three forms that we described in Section2.2,i.e.throughput, resptime,or hybrid.In each case,the shapes of the yield functions for three service classes are the sameother than the magnitude.We determine the ratio of such magnitudes to be4:2:1.The deadline is set to be2seconds.In the case of hybrid,the drop penalty is set to be half of the full yield and the pre-deadline is set to be half of the absolute deadline.Figure6 illustrates the yield functions when they are in each one of the three forms.The request arrival intervals and the query words for the three Differentiated Search classes are based on a one-week trace we collected at Ask Jeeves search via one of its edge web servers.Figure7shows the total and non-cached search rate of this trace.The search engine employs a query cache to directly serve those queries that have already been served before and cached.We are only concerned with non-cached requests in our evalua-tion because only those requests invoke the index search component.We use the peak-time portion of Tuesday,。
容器云平台灾备建设方案目录容器云平台灾备建设方案 (1)一、建设背景 (3)二、需求分析 (3)三、技术路线选型及难点分析 (5)1、现有IT架构和业务架构的痛点 (5)2、容器云建设难点分析 (6)3、容器云技术路线选型 (6)4、厂商选型 (7)四、建设方案 (10)1、总体架构 (10)2、容器云平台灾备 (11)2.1 部署架构 (11)2.2 切换架构 (11)2.3 网络切换 (12)4、环境规划 (16)五、实施经验及效果 (16)1、实施经验总结 (16)2、实施效果 (17)3、建议实施落地原则 (18)1、IaaS云平台建设: (18)2、PaaS云平台建设: (19)3、云管理系统建设: (19)【简介】本文以中小银行数字化转型为背景,对中小银行传统应用迁移容器云平台的实践经验进行总结,探索出适合中小银行特有金融架构特征的容器云平台建设路线。
从业务需求出发,通过建设以容器云平台为基础的底层IT资源平台,为业务发展提供安全、稳定、可靠、灵活的支撑。
同时,本文也将针对容器云平台落地面临的容器灾备、容器云网络建设等问题进行选型和实践经验分享。
一、建设背景近年来,随着金融业务不断扩展,云计算技术在金融行业的发展已经经历过了第一代虚拟化、第二代资源池化,正在向以容器、微服务、DevOps为关键技术和特征的第三代云计算技术前进,以满足金融业新型业务对快速部署、弹性扩展、自动化运维等核心需求。
金融行业已经步入以容器为核心的第三代云计算技术的时代,目前国内大型金融机构虚拟化技术相对成熟,从国有五大行到区域银行都在积极向基础设施云推进,但中小银行相对缓慢,更多处于云平台的尝试使用阶段。
面对高并发、多频次、大流量的全新业务场景,银行业务系统的响应效率变的越发重要,同时金融业务的服务连续性要求也越来越高。
而我行原有的基础架构平台已不足以支撑银行当前的高速信息化建设及创新发展要求。
如何应对不断升级的互联网业务系统,紧跟大行科技信息化建设的步伐,建设具有中小银行特有金融架构特征的容器云平台变得尤为重要。
A distortion-free data hiding scheme for high dynamic range imagesChung-Min Yu,Kuo-Chen Wu,Chung-Ming Wang ⇑Institute of Computer Science and Engineering,National Chung Hsing University,250Kuo Kuang Road,Taichung 402,Taiwan,ROCa r t i c l e i n f o Article history:Received 18December 2009Received in revised form 8February 2011Accepted 21February 2011Available online 27February 2011Keywords:High dynamic range images Data hiding Distortion-freeMessage embedding Steganographya b s t r a c tIn this paper we present a distortion-free data hiding algorithm which can embed secret messages into high dynamic range (HDR)images.Our scheme provides three significant benefits.First,it enables us to convey secret messages to produce a stego HDR image.When we operate the tone mapping technique to reduce the high contrast to a displayable range,no distortion is encountered between the tone-mapped cover and the stego images.A quantitative measure verifies that histograms of the cover and stego HDR images are correlated with linear dependency.To the best of our knowledge,our algorithm is the first approach in HDR literature that can provide capability of distortion-free data embedding.For the appli-cation of image annotation,the average capacity offered by our method is in the range of 0.12–0.29bits per pixel.Our scheme provides an average capacity in the range of 0.0010–0.0026bits per pixel for the application of image steganography where the stego HDR image preserves an HDR image encoding for-mat which does not cause any suspicion by eavesdroppers.Second,our algorithm performs with adaptive message embedding where pixels conceal different amounts of secret messages based on their homoge-neous representations.Quantitative analysis indicates that our algorithm offers an insignificantly small magnitude of the maximal pixel difference between the cover and stego HDR images.This feature and the histogram distribution of similarity between the cover and stego HDR images increase the difficulty of detecting whether any message is hidden in an HDR image.Third,our scheme is efficient.The time required for message embedding or extraction is in the range of several hundred milliseconds.Our approach belongs to a blind detection where the messages can be extracted without referring to the ori-ginal cover HDR image.We believe our proposed scheme is suitable for applications such as image anno-tation or image steganography.Ó2011Elsevier B.V.All rights reserved.1.IntroductionTransmission of private information through the internet in a secret manner is now more frequent due to the prevalence of com-puter science and the internet.This trend encourages researchers to investigate techniques for covert communication.Besides the system of cryptography,data hiding [11]provides an alternative solution to achieving the goal of covert communication.Data hid-ing is a way of secret communication carried out by using various digital multimedia to convey the critical messages,and therefore the major demand here is for both good imperceptibility and a high embedding capacity.Generally,the object in which we intend to embed the secret message is called the cover object indicating that the secret message has not yet been embedded [19].After it has conveyed the secret message,we refer to it as the stego object.While a variety of media,such as text [1],image [17],audio [8],video [10],3D models [2],or general multimedia [7],can serve as a cover object,the image is the most popular medium that is employed for data hiding.An image data hiding technique is usually evaluated in terms of visual quality and embedding capac-ity.The image data hiding algorithm should maximize the amount of messages that can be conveyed in the cover image,and mini-mize the distortion appearing in the stego image caused by the hidden message [17].In addition,data hiding algorithms can be developed to provide features of reversibility.These data hiding algorithms are referred to as reversible data hiding algorithms [14,18]which allow the receiver to extract the embedded data,and completely restore the cover image without losing any of the information.Going one step further,we can produce a stego image without incurring any image distortion which minimizes the distortion to the extreme.This kind of algorithm is referred to as a distortion-free data hiding algorithm.An intriguing feature of the distortion-free algorithm is camouflage of the stego image;consequently,the stego image will not attract much attention by eavesdroppers when it is delivered to the receiver through a public channel.This property makes it useful for data hiding applications such as medical or military image authentication where the quality of the stego image is strictly required,and/or image annotation where sensitivity of the secret message is strictly confidential.0141-9382/$-see front matter Ó2011Elsevier B.V.All rights reserved.doi:10.1016/j.displa.2011.02.004⇑Corresponding author.Tel.:+886422840497915;fax:+886422853869.E-mail addresses:phd9202@.tw (C.-M.Yu),phd9501@.tw (K.-C.Wu),cmwang@.tw (C.-M.Wang).In recent years,there has been an explosion of interest in high dynamic range(HDR)images[13].The‘‘dynamic range’’of a scene is the contrast ratio between its brightest and darkest parts.In con-trast to low-dynamic range(LDR)images,HDR images represent luminance values usingfloating-point numbers for a scene in order to accurately represent the wide range of intensity levels found in real scenes ranging from direct sunlight to deepest shadows.Fig.1 demonstrates the visual difference between LDR and HDR images. The scene has high contrast ratio because of outdoor as well as in-door landscape.When we directly display the LDR image,we lose the details of the outdoor scene because the luminance is out of the range that is supported in an ordinary device.Similarly,the detail in the indoor scene is not visible when we directly exhibit the HDR image.However,we can visualize both details when the HDR image is processed by the tone-mapping operator.Several image processing software and computer games are developed to support HDR images,and they are becoming increasingly popular in variousfields such as digital photography,computer graphics, movies,videogames,and medical imaging.Unfortunately,research in steganography has not kept pace with the advances of HDR images,even though they are expected to replace the low-dynamic range(LDR)images and become the new image standard.To the best of our knowledge,there has been only very limited data hiding work done on HDR images[3].This work produces stego images with high visual quality that is accept-able to human perception.However,an image distortion is inevita-ble due to the hidden messages.In this paper,we provide a new data hiding algorithm that em-beds secret messages into HDR images encoded with the radiance RGBE format[15].Our scheme takes advantage of encoding secret messages to homogeneous representations inherent in the radi-ance RGBE encoding format which has found widespread use in the image community.The scheme provides three significant ben-efits.First,it enables us to convey secret messages to produce a ste-go HDR image.The tone-mapped cover image and the stego images message embedding.The insignificantly small magnitude of the maximal pixel difference and the histogram distribution of similar-ity increase the difficulty of detecting whether any message is hid-den in an HDR image.Third,our scheme is efficient as the time required for message embedding or extraction is in the range of several hundred milliseconds.Our algorithm belongs to a blind detection where the messages can be extracted without referring to the original cover HDR image.Experimental results have verified the feasibility of our algorithm.This paper is organized as follows.In Section2,we review data hiding approaches for HDR images.We then put forward our algo-rithm in Section3.Experimental results are shown in Section4,fol-lowed by the Conclusion and Future Work in Section5.2.Related worksThis section surveys data hiding approaches for HDR images. We were surprised tofind only one paper in the current literature which presents information for HDR data hiding[3].Since that paper utilizes the radiance RGBE encoding as the cover image, we believe it can provide more insight for developing our proposed algorithm.For reference purposes we will briefly describe the ‘‘distortionless data hiding’’that is misleading in the low-dynamic range images,and the approach to producing distortion-free data embedding using permutation.In the following paragraphs we highlight the HDR format,and review the only HDR data hiding algorithm that we have found.A number of data hiding techniques were proposed which use the LDR image as the cover media to convey secret messages. One of these algorithms uses the title of‘‘distortionless data hid-ing’’[20].This causes misconception because the algorithm pro-vides the capability of‘‘reversibility’’which means that once the secret messages are extracted,the original cover image can be restored.Nevertheless,this type of algorithm generates a stegodifference between LDR and HDR images:the LDR image is directly displayed in thefirst column;the HDR image is directly displayed displays the tone-mapping result of the HDR image.226 C.-M.Yu et al./Displays32(2011)225–236encode an optimal message capacity of up to log2(n!)bits,where n is the number of elements to be arranged.The radiance RGBE encoding format[15],originally known as the Radiance picture format,wasfirst introduced as part of the Radiance lighting simulation and rendering system[16].This encoding format has found widespread use for HDR photography and image-based lighting.Other encoding formats include OpenEXR,LogLuv,etc.In the HDR format,the pixel’s color is pre-sented by four channels that include the red,green,blue,and expo-nent channels.This encoding leads to a feature indicating that the representation of a color is not unique.Therefore,we take this advantage to present a distortion-free data hiding algorithm in this paper.Cheng and Wang proposed an adaptive data hiding approach with authentication for a high dynamic range image[3].To the best of our knowledge,their scheme is thefirst such approach for HDR images using the radiance32-bit RGBE encoding.In the radiance format HDR image,the range of luminance intensity is decided by the8-bit exponent value E.Cheng and Wang’s method uses this advantage to classify the pixels into theflat and boundary areas.This pixel classification enables their scheme to remove the restrictions of afixed size of message embedding at each pixel in order to provide larger embedding capacity with little visual dis-tortion.In their reports,their algorithm achieves embedding capacity in the range of 5.13–9.69bits out of32bits of RGBE encoding.Although their algorithm causes image distortion between the cover and stego image,the PSNR values for the tone-mapped stego images are greater than the30dB that is acceptable to human perception.Our survey indicates that an HDR image data hiding algorithm was presented based on the use of the32-bit radiance RGBE encod-ing,and it causes image distortion because of the hidden message. Due to the advantages of data embedding with the distortion-free manner,we believe it is necessary to develop a distortion-free data hiding algorithm which takes this encoding into consideration.Our algorithm is detailed in the next section.3.Our proposed algorithmThis section presents our proposed methods for embedding a secret message into an HDR image without causing any distortion. The method is referred to as a concise fundamental method(CF). This method is simple and direct with an intuitive manner.The low-dynamic range(LDR)images use8bits to represent each of the primary colors,red,green,and blue,leading to a total of24bits of image representations.In an HDR image encoded with the radi-ance format,a pixel is represented by three primary channels fol-lowed by an exponent channel,resulting in a total of32bits of image representation.In each channel,8bits of storage are used so the value at each channel is in the range of0and255.Without loss of generality,let P(r,g,b,e)represent a pixel en-coded with the radiance format,where r,g,and b represent the pri-mary color channels and e indicates the exponent channel which is based on a power of two with the biased number of128.The color of this pixel is afloating point value which can be derived using the floating point conversion as shown in Eq.(1).Similarly,given a col-or in a pixel with thefloating values(R,G,B),we can convert the pixel into the radiance(r,g,b,e)encoding using the integer conver-sion,as shown in Eq.(2),where max(R,G,B)represents the maxi-mum value in the R,G,and B color components.R¼ððrþ0:5Þ=256ÞÂ2ðeÀ128ÞG¼ððgþ0:5Þ=256ÞÂ2ðeÀ128ÞB¼ððbþ0:5Þ=256ÞÂ2ðeÀ128Þð1Þe¼d log2½maxðR;G;BÞ þ128er¼bð256ÂRÞ=ð2eÀ128Þcg¼bð256ÂGÞ=ð2eÀ128Þcb¼bð256ÂBÞ=ð2eÀ128Þcð2ÞDue to the exponent channel that is introduced in the radiance format,we can derive that there is more than one representation todescribe the color of a pixel.For example,we can apply the divisionoperator with the divisor2for each color channel and increase1tothe exponent channel.Given an original pixel P(r,g,b,e),this divi-sion operator will produce a representation A(r/2,g/2,b/2,e+1)which would give nearly the samefloating-point color value andgive identical color after tone mapping for the original pixel pro-vided that components in the color channels,r/2,g/2,and b/2,stillobey the integer form.Similarly,we can apply the multiplicationoperator with the multiplier2for each color channel and subtract1from the exponent channel.This multiplication operator will pro-duce a representation B(2r,2g,2b,eÀ1)which will give nearly thesamefloating-point color value and also give identical color aftertone mapping for the original pixel provided that components inthe color channel,2r,2g,and2b,are within the legal range be-tween0and255.Since each pixel may contain a number of differ-ent representations,the concise fundamental method we proposetakes advantage of this feature to convey the secret message with-out producing any image distortion.We detail this method in thefollowing paragraphs.3.1.Our embedding methodGiven an arbitrary pixel P(r,g,b,e),we define the homogeneous representation group(HRG)for this pixel as a set of representationswhere every element in HRG describes the pixel color identical toP(r,g,b,e).We use the HRG P with the suffix‘‘P’’to denote thehomogeneous representation group for the pixel P.We define thehomogeneity value(HV P)for this pixel as the number of elementsin the homogeneous representation group.We sort every elementin the HRG according to the value represented in the exponentchannel using the ascending order,and assign each sorted elementan index.This allows us to define a homogeneity index(HI)forevery sorted element in HRG where the HI has the range from0to(HV p)À1.As an example,let P(24,160,52,127)represent a pix-el.Then,the homogeneous representation group of this pixel HRG Pcontains three elements as shown in Table1,where the HRG is ex-pressed as HRG P={(24,160,52,127),(12,80,26,128),(6,40,13,129)}.This pixel has the homogeneity value of3(HV P=3),andthe element(24,160,52,127)is assigned to the smallest homoge-neity index of0(HI P=0)because it has the smallest value in theexponent channel.Accordingly,the element(6,40,13,129)hasthe homogeneity index of2(HI P=2).Note that the blue channelin the element(6,40,13,129)contains an odd value of13whichterminates the possibility of applying the division operator.We re-fer to the blue channel as the dominated channel for this pixel.Depending on particular values of a pixel,there are two special cases that we do not determine a pixel’s corresponding HRG formessage embedding.For a pixel P(r,g,b,e),thefirst case is whenthe pixel values in the primary color and exponent channels areTable1An example of a pixel P has four sorted elements in the homogeneous representationgroup(HRG)with the homogeneity value(HV P=3).Pixel P Homogeneityvalue(HV P)Sorted elementsin HRG PHomogeneityindex(HI P)P(24,160,52,127)3(24,160,52,127)0(12,80,26,128)1(6,40,13,129)2C.-M.Yu et al./Displays32(2011)225–236227all zeros,i.e.,P(r,g,b,e)=(0,0,0,0).We refer to this type of pixel as the‘‘null’’pixel.Note that it might be possible for us to apply the division operator255times to produce the homogeneous repre-sentation group,HRG p={(0,0,0,0),(0,0,0,1),...,(0,0,0,255)}, which has the homogeneity value of256allowing us to embed up to8bits of secret message.However,the embedding will pro-duce a relatively large pixel difference after the message embed-ding(see the analysis of pixel difference described in Section 3.3).Therefore,our method does not embed any secret messages when encountering the‘‘null’’pixel.The second special case occurs when the pixel values in the pri-mary color channels are power of2,or one or two of pixel values is/ are zeros,i.e.,P(r,g,b,e)=(2k||0,2k||0,2k||0,e)where k is an integer satisfying the range of06k67and||represents or nota-tion.We refer to this type of pixel as the‘‘neutral’’pixel.Note that the pixel values cannot be all zeros in three channels,because we have defined this kind of pixel as the‘‘null’’pixel.Note again that if the exponent e is less than or equal to248,we can apply the divi-sion operator up to eight times in order to produce the homoge-neous representation group with the homogeneity value of8.For example,when P(r,g,b,e)=(128,128,128,248),there are eight elements in the HRG p where HRG p={(1,1,1,255),(2,2,2,254), (4,4,4,253),...,(128,128,128,248)}.It might be possible for us to adopt this HRG p to embed3bits of secret message.Unfortu-nately,the embedding will produce a much larger pixel difference which becomes evident from the analysis of pixel difference de-scribed in Section 3.3.Consequently,our method excludes the ‘‘neutral’’pixel from message embedding.It is not difficult to determine the homogeneity representation group for a given pixel.In particular,wefirst apply the multiplica-tion operator with the multiplier2to the extreme before we apply the division operator with the divisor2to the extreme.When applying the multiplication rule,the extreme in the multiplication operator means that the components in the color channel are lar-ger than the maximum value of255.The extreme in the division operator,however,means that the components in the color chan-nel have changed to be with thefloating point form.If we operate MU(multiplication)times of multiplication and DI(division)times of division operator,then the homogeneity value of a pixel P is HV P=MU+DI+1.As an example,given a pixel K(20,16,60,127), we consider the color channel(20,16,30)and apply the multipli-cation operator,at most,two times(MU=2)producing two ele-ments,(40,32,120,126)and(80,64,240,125).We cannot apply the multiplication operator anymore because if we do so,the component480will be larger than the maximum value of255. Similarly,we apply the division operator for two times only (DI=2),producing two elements(10,8,30,128)and(5,4,15, 129).We cannot apply the division operator anymore,because if we do so,the component2.5and7.5are with thefloating point forms.The homogeneous representation group of the pixel K is HRG K={(80,64,240,125),(40,32,120,126),(20,16,60,127), (10,8,30,128),(5,4,15,129)}and the homogeneity value of the pixel K will be HV K=2+2+1=5.Note that the homogeneity value of a pixel has the maximal value of7because the component in the color channel must be in the range of0and255.This also means that the homogeneous representation group contains,at most, seven elements.The homogeneity value of a pixel has the minimal value of1indicating that there is an element in the homogeneous representation group which is the pixel itself.Once we have determined the homogeneous group and the homogeneity value of a pixel K,HV K,we can compute the pixel capacity in bits and denote it as C K,as shown in Eq.(3).The pixel capacity indicates how many bits of secret messages that this pixel can offer to convey secret messages.Certainly,the pixel capacity depends on how many elements are in the homogeneous represen-tation group.C k¼b log2ðHV KÞcð3ÞThe embedding process for the cover pixel K(r,g,b,e)can befacilitated using the homogeneity index table(HIT)as shown in Table2.Given a cover pixel K,we can determine the homogeneity value HV K.Depending on this value,thefirst column of Table2lists numbers of bits that can be conveyed.In the third column,we de-scribe the associate bit pattern of the secret message that can be concealed with respect to different homogeneity indices.By refer-ring to the HIT,we can alter the cover status C(HV K,HI K)which re-cords the status of the cover pixel to the stego status S(HV K,HI0K), indicating that a desired bit pattern of the secret message has been conveyed by the stego pixel.The embedding process is best illustrated by an example shown below.Given a cover pixel K(20,16,60,127),following the exam-ple shown above,we can produce the homogeneous representation group for this cover pixel.Table3shows the sortedfive elements in the group where HRG K={(80,64,240,125),(40,32,120,126),(20, 16,60,127),(10,8,30,128),(5,4,15,129)}.Clearly,the cover sta-tus C(HV K,HI K)=C(5,2)because the cover pixel has the homogene-ity value of HV K=5and,according to the sorted elements in HRG K, it has the homogeneity index of HI K=2.Note that since HV K=5,this cover pixel can embed2bits of se-cret message based on the expression shown in Eq.(3).In particu-lar,by altering the cover status C(HV K,HI K)=C(5,2)to S(HV K,HI0K)=S(5,0),appearing in thefirst row third column,we embed two bits of secret message‘‘01.’’In other words,in order to convey two bits of the secret message‘‘01,’’the stego pixel which we should select is the element in the homogeneous representation group HRG K that has the homogeneity index of HI K=0.Conse-quently,the stego pixel is K’(80,64,240,125).As another example, if we intend to embed two bits of the secret message‘‘10,’’we alterC(HV K,HI K)=C(5,2)to S(HV K,HI0K)=S(5,1)in the second row indi-cating that the stego pixel will be K’(40,32,120,126)which has the homogeneity index of HI K=1.We do not need to take any ac-tion if we intend to embed the secret message‘‘11’’because the cover pixel C(HV K,HI K)=C(5,2)happens to have the homogeneity index of HI K=2,which means that C(HV K,HI K)=C(5,2)and S(HV K,HI0K)=S(5,2).Consequently,the stego pixel is exactly the same as the cover pixel.Note that we will not change the cover status C(5,2)to the stego status(5,4)appearing in thefinal row.This is because,though the cover pixel has the homogeneity value ofTable2Homogeneity index table used to embed secret message into a cover pixel K with different homogeneity values HV K.Number ofbitsconveyedHomogeneityvalue(HV K)Homogeneity index0123456 01NP––––––12‘‘0’’‘‘1’’–––––13‘‘1’’‘‘0’’NA––––24‘‘00’’‘‘01’’‘‘10’’‘‘11’’–––25‘‘01’’‘‘10’’‘‘11’’‘‘00’’NA––26‘‘10’’‘‘11’’‘‘00’’‘‘01’’NA NA–27‘‘11’’‘‘00’’‘‘01’’‘‘10’’NA NA NATable3An example of embedding2bits of secret message into a cover pixel K(80,64,240, 125)with the homogeneity value of HV K=5and cover status C(5,2).Sorted elements inHRG KHomogeneityindex(HI K)Status of stegopixelConveyedmessage (80,64,240,125)0S(5,0)‘‘01’’(40,32,120,126)1S(5,1)‘‘10’’(20,16,60,127)2S(5,2)‘‘11’’(10,8,30,128)3S(5,3)‘‘00’’(5,4,15,129)4S(5,4)NA228 C.-M.Yu et al./Displays32(2011)225–236HV K=5,we assign four patterns to represent two bits of secret message.As a result,we denote‘‘NA’’in thefinal row to indicate that we do not assign any bit pattern.We have described the secret message embedding by referring to the homogeneity index table through an example described above.Observing the homogeneity index table shown in Table2 again,the symbol‘‘NP,’’appearing in thefirst row,means that it is not possible to embed the secret message if a pixel has one homogeneity index.The‘‘–’’symbol represents that the homoge-neity index is out of range.Taking the second row as an example, if a pixel K has two elements in its homogeneous representation group,the homogeneity value of this pixel is HV K=2and the pixel has two homogeneity indices,either HI K=0or HI K=1.Homogene-ity indices that are greater or equal to2are certainly out of range. Similar to the embedding example shown in Table2,the‘‘NA’’symbol means that we do not assign any bit pattern.It is worth mentioning that the secret message of the bit pattern associated with the homogeneity index is not identical even though the same numbers of bits are conveyed with different homogeneity values.For example,(HV K,HI K)=(4,0)in the fourth row third column,and(HV K,HI K)=(5,0)in thefifth row third col-umn,all can convey two bits of secret message;however,the for-mer indicates the secret message with the bit pattern‘‘00,’’but the latter depicts the secret message with the bit pattern‘‘01.’’The benefit of adopting the diverse bit pattern is in order to avoid coin-cident alternation of the homogeneity index when embedding the same amounts of secret messages.Another benefit is that the di-verse bit pattern will reduce changes encountered in the histogram distributions(at red,green,and blue color channels)for the stego HDR image.This avoids the attack of histogram inspection that is commonly employed in the steganalytic technique.We will pres-ent a quantitative measure of the histogram distribution in the experimental results which will demonstrate that our proposed algorithm produces a similar histogram distribution even though a number of secret messages have been conveyed in the stego HDR image.Finally,the homogeneity index table is a necessity both in the message embedding and the extraction.Therefore, we can use a secret key,Key-1,to increase the security,thereby avoiding the attack of eavesdroppers.Given an HDR image with a resolution of MÂN,the message embedding in the concise fundamental method is operated using the following four steps:Step1:We examine every pixel according to a secret key,Key-2, which determines the embedding order of the secret message.For the examined pixel,such as K,we determine the corre-sponding homogeneous representation group(HRG K)and calcu-late the homogeneity value(HV K)of the HRG K.Step2:For the examined pixel,the pixel capacity C K is com-puted using Eq.(3).If the homogeneity value(HV K)is less than or equal to1,this pixel cannot convey any secret message.We go back to Step1and process the next pixel.Otherwise,we read in C K bits of secret messages accordingly.Step3:We compute the current cover pixel status C(HV K,HI K).Based on the secret message,C(HV K,HI K),we determine thedesired stego pixel status S(HV K,HI0K)by referring to C(HV K, HI K),the Homogeneity Index Table,and the secret message.Step4:We alter the current cover pixel K to become the stego pixel K0by selecting an appropriate element in HRG K that hasthe homogeneity index of HI0K.Once this has been done,we can process the next pixel starting from Step1.The total embedding capacity(TMC)of an HDR image can be computed by examining the homogeneity value of each pixel,as shown in Eq.(4).TMC¼XMÂNi¼1b log2ðHV iÞcð4ÞThe extraction of the secret message is straightforward.Given a stego HDR image,we examine every pixel in a specific order de-rived by using the secret key,Key-2.For each stego pixel,such as K0,which we inspect,we compute the homogeneity value HV K0 for this stego pixel.If HV K0is less than or equal to1,then this pixel conveys no secret message.We then process the next pixel in a specific order.Otherwise,we produce the homogeneous represen-tation group,HRG K0for this stego pixel and calculate how many bits of secret message,say SM,are concealed in the cover pixel K0using Eq.(3).By comparing the cover pixel K0with all of the elements in HRG K0,we can determine the homogeneity index of the cover pixel,HI K0,and produce the status of the stego pixel S(HV K,HI0K).Given the secret key,Key-1,we can produce the homogeneity index table (HIT).Finally,we can extract SM bits of secret message by referringto HIT and S(HV K,HI0K).This ends the extraction of a stego pixel K0, and we can proceed to extracting the secret messages concealed in the next stego pixel.3.2.Pixel categories classificationThe proposed method embeds secret messages into the HDR images.In this section,we further discuss the issue of pixel classi-fication in order to provide an insight for the pixel distribution in a single HDR image.In addition,the pixel classification can illustrate pixels that are eligible for message embedding.We classify the pixels in an HDR image into totally seven cate-gories,as shown in Table4,where max(r,g,b)represents the max-imal values of a pixel for the three channels.Whenever possible, we illustrate an example of a pixel in each category.Based on its features,a pixel is classified into either‘‘regular’’or‘‘irregular’’pixels,as shown in Table4.The‘‘regular’’pixel affirms that the maximal values of a pixel for the three channels,abbreviated as max(r,g,b),is equal to or greater than128.In contrast,the‘‘irreg-ular’’pixel represents that max(r,g,b)6127.Note that a pixel can belong to one of these two categories(‘‘regular’’or‘‘irregular’’),but not both.Based on the point of view of message embedding,a pixel can be classified intofive categories.Pixels belonging to thefirst twoTable4The categories of pixels based on two classification bases in an HDR image encoded by the RGBE format.Classification basis Pixel category Satisfied conditions Example of a pixel P(R,G,B,E)Pixel features Regular max(r,g,b)P128P(12,19,132,131)Irregular max(r,g,b)6127P(127,43,64,133) Message embedding Embeddable26HV P67P(12,132,26,134)Promising max(r,g,b)=127(HV P=2)max(r,g,b)=254and r,g,b2even(HV P=2)P(43,127,56,125) P(254,142,38,129)Singular HV P=1P(129,124,122,130)Null r=g=b=0(HV P>8)P(0,0,0,128)Neutral HV P=8P(128,128,128,248)C.-M.Yu et al./Displays32(2011)225–236229。
持续交付中的容器化部署技巧随着软件开发和部署的日趋复杂,传统的软件传递方法已经无法满足快速交付的需求。
在这种情况下,容器化部署技术应运而生。
容器化技术通过将应用程序及其所有的依赖项打包成一个独立的、可移植的容器,从而实现了快速、可靠和一致性的部署。
本文将讨论一些在持续交付中使用容器化部署的技巧。
一、选择适合的容器技术在开始容器化部署之前,选择适合的容器技术非常重要。
目前最流行的容器引擎是Docker,它提供了方便的命令行工具和图形界面,使得容器的创建、管理和部署变得非常简单。
Docker有一个庞大的用户社区,提供了大量的镜像和工具,可以满足各种不同的需求。
二、创建高效的容器映像一个好的容器映像是成功容器化部署的关键。
在创建容器映像时,要尽量减小映像的大小,因为映像的大小直接影响部署时间和网络传输速度。
可以通过使用多层构建、移除不必要的依赖项和优化文件系统来减小映像的大小。
此外,容器映像的版本控制也非常重要,建议使用版本控制系统(如Git)来管理映像的变化。
三、使用容器编排工具在实际部署中,往往需要同时管理多个容器。
使用容器编排工具可以简化这个过程,帮助我们定义和管理容器之间的关系和依赖关系。
例如,Docker Compose可以帮助我们通过一个单一的配置文件定义和运行多个容器。
而对于大规模的容器集群管理,Kubernetes是一个更好的选择,它提供了强大的自动扩容、负载均衡和服务发现功能。
四、持续交付和自动化测试持续交付要求各个环节都要自动化,包括容器创建、部署和测试。
在容器创建方面,可以使用自动化工具(如Jenkins)来自动构建和创建容器映像,并将其推送到仓库中。
在部署方面,使用自动化工具可以大大减少人工错误,并提高部署的一致性和可重复性。
而在测试方面,可以使用自动化测试工具(如Selenium)来运行集成测试和回归测试,以确保应用程序在不同环境中的稳定性和一致性。
五、监控和日志管理容器化部署后,亦需对容器进行监控和日志管理。
使用容器技术实现高可用性和容错性的应用部署随着云计算和微服务架构的发展,容器化技术成为了现代应用部署的新宠。
容器技术通过将应用及其依赖打包成容器镜像,实现了应用在不同环境中的快速部署和迁移。
然而,容器技术不仅仅是为了实现快速部署,更重要的是它可以为应用提供高可用性和容错性的支持。
首先,容器技术提供了弹性的资源管理机制,可以根据应用的负载动态调整资源的分配。
一方面,通过集群管理器,可以根据应用的需求自动调整容器的数量,以应对高峰期的访问流量。
另一方面,容器技术还支持自动加载均衡器,以实现流量在容器间的均衡,从而提高应用的可扩展性。
通过这种方式,即使某个容器出现故障,其他容器仍然可以继续提供服务,从而提高了应用的可用性。
其次,容器技术还提供了容器编排工具,可以实现应用的高可用性和容错性。
容器编排工具可以根据事先定义的策略,自动地监测和管理容器集群中的容器。
当某个容器出现故障时,容器编排工具会自动将故障容器重新部署到其他健康的节点上,以实现容器的自愈能力。
同时,容器编排工具还可以监测应用的状态,当应用出现异常时,可以自动地进行问题排查和修复,从而提高了应用的容错性。
另外,容器技术还支持多节点部署,通过将应用的容器分布在不同的节点上,可以实现应用的高可用性。
当某个节点出现故障时,其他节点上的容器仍然可以继续提供服务,从而保证了应用的可用性。
与传统的部署方式相比,容器技术的高可用性是基于容器的自愈能力和容器编排工具的支持,不依赖于硬件设备的冗余和负载均衡器的配置,从而降低了部署和维护的成本。
此外,容器技术还提供了快速迁移和回滚的能力。
容器可以被打包成镜像,通过镜像的导出和导入,可以实现应用在不同环境中的快速迁移。
这种特性不仅可以提高开发环境和生产环境之间的兼容性,也可以实现多云环境下的应用迁移。
而且,容器技术还支持版本管理,通过版本控制工具可以快速回滚到之前的版本,从而提高了应用的可靠性。
综上所述,使用容器技术可以实现高可用性和容错性的应用部署。
如何构建高可用架构随着互联网的快速发展和更新换代,越来越多的企业意识到构建高可用架构的重要性。
在星际时代,任何一点故障都将造成无限的影响,因此,企业必须有足够的技术能力来应对。
构建一个高可用架构的最高目标是确保100%的应用程序可用性,来应对所有的计算需求。
然而,构建高可用架构并不容易,需要使用一系列技术和方法才能确保其成功。
本文旨在通过分析高可用架构要素和相关技术进行探讨,帮助企业构建出稳定的高可用架构。
一、高可用架构要素高可用架构确保系统的可用性并最小化停机时间,同时保证了系统的高性能。
因此,高可用架构包括以下要素:1.数据可用性数据是企业最重要的资产之一,因此,确保其可用性至关重要。
高可用架构要求在数据库中部署数据备份和恢复策略,同时在部署过程中考虑数据的复制和同步方式,以确保数据的可用性。
此外,定期测试数据配合策略也是非常重要的。
2.服务可用性服务可用性是高可用架构的另一重要要素。
构建稳定和高效的服务器和网络设施是最为基础的。
企业必须考虑如何平衡服务器负载,避免出现瓶颈问题,保证服务的可用性。
3.应用程序可用性应用程序是企业的关键业务流程,必须保证其可用性。
高可用架构需要考虑多台服务器和分布式架构的应用程序效率和负载均衡,确保其可靠度和时效性。
4.自动化构建高可用架构的关键之一是实现自动化,包括软件部署、配置管理和变更管理等。
为了提高可用性和灵活性,企业必须自动化部署和管理重要应用程序和系统配置,以降低人为错误的发生率和实现更及时的响应。
二、高可用架构相关技术1.负载均衡对于每个运行中的应用程序来说,如何平衡不同的服务器负载是最重要的。
负载均衡可用于分散服务器上的负载,实现动态路由,确保服务可用性。
企业可以采用不同的负载均衡技术,比如DNS、软件或硬件负载均衡,来维持高可用架构,确保应用程序的稳定性和可用性。
2.故障转移故障转移技术可以确保应用程序在服务器出现故障时自动重定向到健康的服务器上。
nerdctl restart策略nerdctl是一个用于与容器运行时接口(CRI)兼容的容器运行时。
它是Docker CLI的一种替代品,使用类似于Docker CLI的命令进行容器管理。
nerdctl支持包括重启策略在内的各种容器管理功能,为用户提供了更灵活、高效的容器操作体验。
重启策略是指在容器发生故障时,系统如何自动将容器重启起来。
nerdctl支持多种重启策略,可以根据具体情况选择适合的策略来保证容器的高可用性和稳定性。
nerdctl支持的重启策略有以下几种:1. no重启策略:当容器停止时,不会自动重启。
这是默认的重启策略。
2. always重启策略:当容器停止时,会自动重启容器。
这种策略适用于需要保持容器一直运行的场景,比如Web服务器。
3. on-failure重启策略:当容器非正常停止(退出状态码非零)时,会自动重启容器。
可以通过设置--restart flag来指定重启次数,比如--restart=on-failure:5表示在容器失败5次后停止重启。
4. unless-stopped重启策略:容器停止时,除非手动停止容器,否则会自动重启。
这种策略适用于需要容器一直运行,但允许用户手动停止容器的场景。
重启策略是通过在运行容器时使用--restart flag来指定的。
以下是一些使用示例:1.使用no重启策略:```nerdctl run --restart=no my-container```2.使用always重启策略:```nerdctl run --restart=always my-container```3.使用on-failure重启策略:```nerdctl run --restart=on-failure:5 my-container```4.使用unless-stopped重启策略:```nerdctl run --restart=unless-stopped my-container```通过合理选择重启策略,可以在容器发生故障时快速恢复服务,并确保容器的高可用性和稳定性。
通过Docker实现容器化的持续交付流程随着云计算和DevOps的兴起,软件交付过程中的持续集成和持续交付变得越来越重要。
在这个过程中,Docker已经成为了一种非常流行的工具,它可以帮助开发人员在不同平台上实现容器化的持续交付流程。
容器化是将应用程序及其依赖项打包在一个独立的、可移植的容器中的过程。
通过使用容器化技术,开发人员可以将应用程序与其所依赖的库和配置文件一起打包,并在不同的环境中运行,而不需要担心环境差异性带来的问题。
Docker是一种开源工具,它可以自动化部署和应用程序的管理。
通过将应用程序和其依赖项打包在一个容器中,Docker可以提供可靠、一致和可重复的环境,从而实现快速的开发和部署。
容器化的持续交付流程通常包括以下几个关键步骤:1. 定义Docker镜像:Docker镜像是一个只读的文件,它包含了应用程序的代码、环境和依赖项。
开发人员可以使用Dockerfile来定义镜像的构建过程,包括基础镜像的选择、软件安装和配置等。
2. 构建镜像:通过使用Docker命令行工具或者CI/CD工具,开发人员可以自动化地构建Docker镜像。
在构建过程中,Docker会根据Dockerfile的定义,从零开始创建一个包含完整应用程序的镜像。
3. 测试镜像:构建完成后,开发人员需要对镜像进行测试,以确保其可靠性和一致性。
这包括运行自动化测试、进行安全扫描和性能测试等。
4. 分发镜像:一旦测试通过,开发人员可以将镜像发布到镜像仓库中,供其他开发人员和部署团队使用。
这样,其他团队可以在需要时,从镜像仓库中拉取镜像并部署到不同的环境中。
5. 部署容器:使用Docker容器编排工具,如Docker Compose或Kubernetes,开发人员可以轻松地部署和管理容器化的应用程序。
容器编排工具为开发人员提供了更高级的功能,如自动扩展、负载均衡和服务发现等。
通过上述步骤,开发人员可以实现一个完整的容器化持续交付流程。
学习计划整体课程持续 16 周时间课程内容第一周测试开发体系介绍、编程语言第二周Pytest 测试框架与 Allure 测试报告第三周用户端 Web 自动化测试第四周用户端 App 自动化测试第五周用户端自动化测试第六周接口测试第七周接口自动化测试第八周性能测试第九周安全测试第十周持续集成/持续交付第十一周Docker 容器技术与 K8S第十二周测试框架开发第十三周测试平台开发第十四周测试平台开发进阶第十五周算法与设计模式第十六周面试与简历线下面授安排面授安排主课:包含对应课程的录播与直播,需要完成配套作业。
赠课:提供对应课程的录播与直播。
直播训练营测试开发进阶学习路线Web 测试用例设计赠课App 测试用例设计赠课测试流程实战赠课Linux 命令与 Bash 脚本赠课Python 或 Java 编程语言赠课单元测试与集成测试框架主课用户端(Web)自动化测试-企业微信主课用户端(Web)自动化测试-飞书主课用户端(App)自动化测试-企业微信主课用户端(App)自动化测试-雪球主课专项测试赠课小程序测试赠课接口协议抓包分析主课接口自动化测试-企业微信主课接口自动化测试-飞书主课性能测试主课安全测试主课算法与设计模式主课持续集成/持续交付主课Docker 容器技术与 K8S主课测试框架开发主课测试平台开发主课测试左移/白盒测试/单元测试/覆盖率/代码审计赠课测试右移/性能监控/质量监控赠课测试开发体系介绍测试体系介绍-L1教学目标掌握软件测试的基础概念了解测试行业经典的测试方法与经典书籍知识点形式章节描述知识点软件测试基础概念软件测试的原则、对象、作用,缺陷的介绍知识点软件开发流程SCRUM、XP、持续集成/持续交付/DevOps知识点测试流程体系传统测试流程、测试左移、测试右移知识点测试技术体系分层测试体系、单元测试、UI 测试、接口测试、白盒测试知识点常用测试平台用例管理平台、Bug 管理平台、代码管理平台、持续集成平台知识点流程管理平台常用流程管理平台介绍,jira、redmine知识点项目管理与跨部门沟通协作与产品经理、研发、上下游测试的配合知识点测试经典书籍拆分讲解全程软件测试、探索式测试、持续交付、Google 测试之道、不测的秘密等测试体系介绍-L2教学目标掌握测试方案设计方法。
方案整体架构描述概述本文旨在描述一个具有可扩展性和高可用性的系统的整体架构设计方案。
系统的主要功能是提供一系列在线服务,包括但不限于用户管理、商品展示、订单处理等。
架构设计技术选型系统主要使用以下技术:•Spring Boot作为应用框架•MySQL作为数据库•Redis作为缓存•RabbitMQ作为消息队列•NGINX作为反向代理架构图下图展示了系统的整体架构:+---------+ +------------+| Web 客户端1 | |+---------+ | || NGINX 负载均衡器 |+---------+ | || Web 客户端2 | |+---------+ +------------+||v+------------+| Spring Boot 1 | <----> MySQL 1 +------------+|| RabbitMQv+------------+| Spring Boot 2 | <----> MySQL 2 +------------+|| Redisv+------------+| Spring Boot 3 |+------------+如图所示,系统主要由以下部分组成:•客户端:用于提供Web界面和API服务,分布在不同的地区•NGINX负载均衡器:用于分发请求到不同的客户端•Spring Boot:主要用于提供各种微服务和后端API服务,其中Spring Boot 1和Spring Boot 2用于提供具体的服务,Spring Boot 3用于提供监控和管理服务•MySQL: 用于存储各种业务数据•Redis: 用于缓存数据,提高系统响应速度•RabbitMQ:用于异步消息传递,增加系统可扩展性系统分层为了便于管理和维护,系统采用分层架构,包括以下四个层次:•表现层:负责接收并处理用户的请求,返回相应的数据•业务逻辑层:负责处理各种业务逻辑,包括但不限于用户验证、订单处理、商品展示等•数据访问层:负责访问各种数据源,包括MySQL和Redis•基础设施层:负责提供各种基础设施服务,如邮件服务、短信服务、日志服务等每个层次的职责都非常清晰,便于开发人员的协作和系统维护。
智简智慧园区网络解决方案技术创新,变革未来目录1.园区网络的趋势和挑战2.智简园区网络,引领园区进入Gbps全无线时代•方案架构•规划自动化•建设自动化•运维&调优智能化3.智简园区网络产品全景园区网络进入无线化、物联化、云化的时代有线升级无线AVR 教学多媒体教学无线化高教全无线办公无线园区Welink 无线办公大企业电子价签无线移动收银自助结账机无线磅秤零售商超生产自动化AGV 资产管理高清质检大众汽车海量分支管理远程运维物联网价签海量分支公共Wi-Fi MISE 无线城市政府公共西交大平安科技意大利MISE园区网络业务模型变化和挑战网络规模增长质量严苛应用增加终端数量增长300亿2020年全球IoT 联接数量数据来源: IoT Analytics, Gartner83%WLAN 物联终端未来3年增长率•物流AGV 零丢包•无线投屏时延< 50 ms •自助付款10X 可靠性•视频监控0中断•网络规模增长X 倍•应用类型增加Y 倍•网络管理人员仅增加1.X 倍挑战如何满足万物互联的带宽挑战如果保障应用体验挑战如何快速开通网络和降低OPEX目录1.园区网络的业务场景、趋势和挑战2.智简园区网络,引领园区进入Gbps全无线时代•方案架构•规划自动化•建设自动化•运维&调优智能化3.智简园区网络产品全景Wi-Fi 6时代,全无线智简园区网络TelemetryNETCONF/YANGMulti-GE25GE/40GE 100GEAirEngineCloudEngine高品质承载全无线园区网自动驾驶网络管理与控制系统开放的行业应用开发平台客流分析电子书包健康管理智慧办公SDK | API管理、控制、分析打造高品质承载的全无线园区网络室内高密物联密集机器互联房间密集室外高密接入交换机汇聚交换机S5732-H 多速率S5732-H 光电混合宽温交换机S6730-S S6730-HS12700E核心交换机全无线接入Gbps-Ready 的极速无线:•满足全场景的Wi-Fi 6接入•AirEngine 独家16T16R 天线,提供10.75Gbps 每AP ,1.6Gbps 每用户峰值性能高密100G 的无损核心:•CloudEngine S 交换机,57.6Tbps 的交换容量,一台顶6台带宽弹性的园区承载:•独家无线空间流RTU ,独家多速率以太RTU•业界唯一光电混合,支持10Gbps 300米POE++连接基于SDN 和AI 打造的园区网络自动化管理和运维的架构全场景全融合WAN简单业务园区多业务园区多园区互联管理、控制、分析全生命周期规划建设运维优化TelemetryNETCONF/YANG一个平台实现对网络的管理、控制和分析同时服务于WLAN 、LAN 和WAN 网络提供全生命周期的网络服务智简园区网络的3大客户价值智能运维网络分钟部署故障分钟修复15mins/站点整网即插即用主动发现85%故障智能无线射频调优智能超宽智能联接每用户无线超Gbps 速度863Mbps1.6GbpsWi-Fi 5Wi-Fi 616T16R 智能天线网络带宽弹性升级GE 2.5G 5G 10G业界独家MGE RTU Pay-as-you-grow随时随地100Mbps 覆盖应用无损的体验100Mbps 连续组网独家OFDMA+MU-MIMO 联合调度用户视频会议邮件Web 下载业界独家用户+应用精细化调度重构网络的规建维优,来实现园区网络的极致体验和极简运维①规划②建设③运维④优化依赖工程师经验自动化工具人工调优智能调优网络管理AI智能运维SDN自动化命令行配置•无线组网规划•有线组网规划•数据业务设计•软硬件安装•设备配置•无线网络•有线网络•应用质量•日常监控•例行维护•故障定位与修复省时间+省时间+省时间+省时间+ 体验优化体验优化省钱省钱目录1.园区网络的业务场景、趋势和挑战2.智简园区网络,引领园区进入Gbps全无线时代•方案架构•规划自动化•建设自动化•运维&调优智能化3.智简园区网络产品全景园区网络的3种典型场景简单业务园区多业务园区多园区互联WANHQ分支特征•业务复杂,配置多•一个园区一个模型•业务简单,配置少•站点数量多,但可提炼模型•多个站点,比如总部+分支•站点间有互访需求举例华为手机销售门店普教校园网络高校校园网络大型企业园区华为总部和各分支机构大企业网络规划要做哪些事?WLAN 网络规划有线网络规划数据业务规划•AP 数量、位置•功率、信道、频宽•设备选型•组网方案•组网拓扑•设备选型•基础业务(IP/VLAN 等)•接入控制•WLAN 业务(SSID )•网络安全•QoS 设计专业工程师现场工勘、基于经验设计规划内容传统规划方式专业工程师基于经验设计专业工程师基于经验设计当前的网络规划,离不开有经验的专业工程师网络规划哪些事可以做到自动化减少专业人力投入?WLAN 网络规划有线网络规划数据业务规划•AP 数量、位置•功率、信道、频宽•设备选型•组网方案•组网拓扑•设备选型•AAA•WLAN 业务(SSID)•网络安全•QoS 设计WLAN 云网规预置模型自动设计网络资源统一规划自动分发•基础业务(IP/VLAN 等)通过云端工具实现WLAN 网规和仿真智简园区网络通过云化工具和云管理实现部分场景的规划自动化规划内容针对小型简单业务园区,通过预置模型实现自动组网和业务设计针对多业务园区,在控制器统一规划网络资源,网络建设时自动分发网络资源自动化工具WLAN云网规,30分钟完成万人园区无线规划环境设置区域设置自动布放AP 信号仿真1 32 4•支持室内、室外两种场景•5步,30分钟即可完成万人园区无线规划•支持3D信号仿真和3D漫游仿真,直观感知规划效果https:///serviceturbocloud/#/WLAN云网规网络资源规划预置模型业界首创3D信号仿真,直观感知覆盖效果WLAN云网规网络资源规划预置模型业界首创3D 漫游仿真,提前判断漫游效果①预设漫游路径最强AP AP-25 CH: 6/36场强 2.4G: -49.7dBm 5G: -47.5dBm 应用层速率2.4G :138Mb/s 5G :135Mb/s②展示终端当前连接AP ,接入速率和无线信号信息WLAN 云网规网络资源规划预置模型针对小型简单业务园区,通过预置模型实现自动设计华为终端门店自动有线和无线组网定义自动数据业务定义根据场景选择,推荐组网模型和数据业务设计举例:零售门店、经济型酒店、普教•单AP •AR+AP•AR+交换机+AP •……•员工SSID •访客SSID •IP 、VLAN •静态路由预置多种网络模型:基于华为项目经验,针对简单业务园区,提炼组网和业务配置模型,并预制在NCE-Campus简单业务园区(小型)如何实现自动组网和业务设计特征1:组网简单,设备少特征2:业务简单,配置少特征3:数量多,可提炼模型WLAN 云网规网络资源规划预置模型针对多业务园区,在控制器统一规划资源,自动分发举例:高校、政企大楼等多业务园区某大学校园在控制器统一规划网络资源•网络资源线上规划,在网络建设时,自动分发•通过GUI 视图编排,简易快捷特征1:组网复杂,设备多特征2:业务复杂,配置多特征3:一个园区一个模型WLAN 云网规网络资源规划预置模型园区网络规划自动化全景有线网络规划WLAN云网规简单业务园区多业务园区多分支互联园区园区网络规划自动化WLAN网络规划数据业务规划预置模型自动设计预置模型自动设计网络资源云端规划自动分发预置模型自动设计WLAN云网规网络资源池自动分发•IP、VLAN •AAA•WLAN业务(SSID)•网络安全•QoS设计NA NA各分支同简单业务园区和多业务园区各分支同简单业务园区和多业务园区各分支同简单业务园区和多业务园区小型中大型中大型目录1.园区网络的业务场景、趋势和挑战2.智简园区网络,引领园区进入Gbps全无线时代•方案架构•规划自动化•建设自动化•运维&调优智能化3.智简园区网络产品全景数字化时代,组织的业务开通和发放大大加速1家医院10天建成100+数字化系统2400家门店1年新开3天建成网络传统网络建设方案无法满足加速的业务开通和发放传统网络建设的3个特征网络开通,采用命令行烟囱式网络,专网专用业务策略,静态预设安防网办公网生产网园区主干网香港X大学,办公、监控、IoT等10余张网络,专网开通,7人团队部署周期30天某零售超市,新部署有线和无线网络,60台设备,安装3天、设备配置耗时2周某企业办公园区,4000用户,网络访问策略调整,耗时2-3周技术变革带来了自动化,加速网络开通虚拟办公网络虚拟物联网络虚拟监控网络通过SDN 和云管理实现即插即用物理网络CLI去CLI通过Overlay 实现一网多用和安全隔离专网专用一网多用静态预设动态发放通过SDN 实现自动化策略发放虚拟网络业务策略接入访问QoS带宽组1员工组2访客组1VIPX√如何实现设备的即插即用?物理网络业务策略虚拟网络业界首创APP扫码开局,2分钟完成设备即插即用。
持续交付中的容器化部署技巧随着软件开发和交付方式的快速发展,持续交付(Continuous Delivery)成为了许多企业实现快速、高质量交付的重要方法。
而在持续交付的过程中,容器化部署技巧的应用更是成为了不可忽视的一部分。
本文将探讨持续交付中的容器化部署技巧,探索如何更好地应用容器化技术来实现持续交付的目标。
一、容器化技术简介容器化技术是一种虚拟化技术,通过将应用及其所有依赖打包成一个独立的运行环境,实现应用在不同环境中的快速部署和移植。
常见的容器化技术包括Docker和Kubernetes等。
这些技术提供了快速、轻量级的部署方式,同时也保证了应用的安全性和便携性。
二、容器化部署的优势1. 极简部署:通过容器化技术,开发人员只需打包应用及其依赖,即可在不同环境中进行部署。
无需关注底层系统的差异,大大简化了部署过程,提升了部署效率。
2. 快速伸缩:容器化环境下,可以根据实际需求快速扩展或缩减应用容器的数量。
只需通过调整容器数量的配置参数,即可实现自动化的伸缩效果,从而满足高并发场景下的需求。
3. 环境隔离:每个容器拥有独立的运行环境,相互之间隔离,互不干扰。
这种隔离性可以有效减少因为环境差异导致的问题,同时提供更高的安全性。
4. 快速回滚:容器化部署技术支持快速回滚功能。
当部署出现问题时,只需切换到上一个版本的容器即可恢复系统,减少对业务的影响。
三、容器化部署的关键技巧1. 资源管理:在使用容器化部署时,合理管理资源是关键。
需要对应用的内存、CPU等硬件资源进行合理配置,以避免资源浪费和性能瓶颈。
同时,基于实际情况进行资源伸缩的配置,可以提升系统的弹性和灵活性。
2. 容器编排:容器编排工具如Kubernetes可以帮助我们管理大规模容器集群。
通过定义Pod、Service和Deployment等资源对象,可以自动化部署、扩缩容和健康检查等操作。
合理使用容器编排工具,能够有效提升容器化部署的效率和可靠性。
持续部署,并不简单!Java 分页原理与实践(上)MongoDB复制集—快速搭建复制集使用wordpress极速建站python正则表达式陈皓注:本文由@常新居士完成初稿,我做了一些编辑,主要写了第五节“没完”。
这几年,持续集成随着敏捷在国内的推广而持续走热,与之相伴的持续部署也一直备受关注。
自前两年,持续交付这个延续性概念又闯进了国内IT圈,慢慢开始在社区和会议中展露头角。
许多不明真相的群众跟风哭着喊着要“上”,而许多前CI的半吊子玩家换件衣服就接着干,有的甚至衣服都来不及换……国内的这些土财主如果不巧请了某些所谓的战略家,除了建了一堆持续集成环境,以及每天嚷嚷着要这个要那个,混乱的状况在根本上没有得到改善。
本文无意费力探讨持续集成和持续交付的概念,而是打算谈谈对于大型软件企业,以持续集成为基础实现持续部署(交付)时,所要面对的问题以及可行的解决方案。
地主老财们,夜黑风正猛,山高路又远,注意脚下……And God Said, Let there be light: and there wa— GENSIS, Charpter 1, King James一、起步先来讲个故事……几年前,一对留美的夫妇通过朋友找到我,让我帮忙在国内组建一个开发团队,该团队负责为其开发一款基于社交网络的客户关系管理软件,(暂且称之为项目A)。
这个项目除了尚不清晰的需求范围和很紧的期限外,作为业内人士的老公Richard根据眼下流行的软件开发过程还提了诸多额外的要求:●功能要及早交付(以便拿去和潜在的投资人洽谈)●功能在部署到生产环境前要先部署的一个测试环境(Richard要试用后给予反馈)●功能必须经过测试(长期作为软件外包的甲方,对质量要求严格)●要减少后期维护的工作(美国人精贵,少雇一个是一个)●支持协同开发(以便维护人员及早介入)●……这正是持续集成所要解决的典型场景。
针对Richard的要求,我们只要建立一个基于Hudson(现在叫Jenkins)+Maven+SVN 的持续集成环境(再加上持续集成所要求的测试和过程)就可以很好地满足上述要要求,此方案的结构如下:对于上述方案,让我们近距离看看各个服务器的内部情况,以及人员在这种方案下的分工协作:我们先谈谈上面的图中涉及的一些概念性问题:1.1)编译时依赖和运行时依赖从字面上不难理解这两种依赖的类型。