A Linux Based Software Router Supporting QoS, Policy Based Control and Mobility

A Linux Based Software Router Supporting QoS, Policy Based Control andMobilityJayachandran Maniyeri, Zhishou Zhang,Radhakrishna Pillai.RInstitute for Infocomm Research, 21 Heng Mui Keng Terrace,Singapore 119613^mjay, zszhang `@.sg,krishna@iimk.ren.nic.inPeter BraunSiemens AG, PG U SE D4, Hofmannstr.51, D-81359 Munich, Germany peter-m.braun@icn.siemens.deAbstractIn ord er to accommod ate the new generation of Internet applications, routers need to be d esigned to support QoS, policy based management, and mobility. This paper presents an architecture for the control plane of a software router that integrates signaling protocols an dcontrol mechanisms for QoS, policy base dmanagement, and mobile IP. The architecture has the flexibility of using policy control to dynamically configure routers to act as nodes in an IntServ or DiffServ domain. Within each d omain the routers can be configured to control their resource usage or to implement Service Level Agreements. Implementation of a software router base d on Linux an d the experimental results are presented . The results show that with software routers this architecture can be used to meet the end-to-end QoS performance for most Internet applications.1IntroductionTraditional IP-based networks provide only ‘best effort’ data delivery, which may result in packet loss, longer communication delays and larger jitter (variability in delivery delay) than what is acceptable for many new generation Internet applications like VoIP and VoD. A possible solution is to provide QoS for certain users, services and applications through preferential treatment of their traffic over others at the network elements, for e.g., an application, host or router. Integrated Services (IntServ) and Differentiated Services (DiffServ) are two models standardized by the IETF to realize QoS in IP networks. Current belief is that a single model will not suffice to support end-to-end QoS to applications over heterogeneous networks. The work presented here isbased on the IntServ-DiffServ architecture, with IntServ at the access network and DiffServ at the core network [1]. End-to-end QoS is realized through flow based admission control at the access network using RSVP [2] and DiffServ forwarding in the core network.Policy Based Network Management (PBNM) is a dominating technology for automated and business objectives driven management of network resources. QoS enabled networks need policies to distribute the available resources based on the business goals of the network provider. Policies encode the high-level goals and requirements of management. Policy based management can also be adopted for other domains like security and Virtual Private Networks. The interface to the network device and the information models required for specifying policies are either standardized or being standardized in IETF and DMTF. Details on the PBNM system used in this project can be found in [3].Wireless LANs, Mobile IP [4] and mobility management are areas of growing interest. Mobile users roaming in foreign networks with their laptops or PDAs will likely be a future trend. Roaming mobile users require the same services as they would get when attached to their office LAN.In order to accommodate these recent trends, Internet routers need to be designed to support QoS, policy based management, and mobility. This paper presents the router control plane architecture for such a router from the QoS/PBN 1 project. The architecture is experimented in an IP network supporting IntServ-DiffServ QoS with mobile IP support and with PBN management. The network is set up using software routers based on Linux and the performance results are reported. One of the main1QoS/PBN is a joint R&D project between Siemens AG and Institute for Infocomm Research, Singapore.contributions of the QoS/PBN project is the evaluation of an integrated architecture for the new generation of routers. An experimental platform has been set up using available implementations of various software components. The system also uses a standards based policy decision engine that was developed as part of this project. Refer to section 4 for details on the software components used.This paper is organized as follows. Section 2 presents related literature. The prototype experimental network that introduces the different types of routers in IP networks is described in Section 3. The architecture of the router control plane is explained in detail in Section 4 and the performance results are provided in Section 5. Section6 concludes the paper.2Related literatureAlmesberger et al. [5] describe a Linux based implementation of IntServ operation over DiffServ networks. The main focus of this work is on the qualitative mapping of the IntServ Controlled Load Service at the access network into DiffServ Expedited Forwarding "Per Hop Behavior" at the core of the network. IntServ network uses RSVP for signaling QoS requirements. A more quantitative mapping of QoS, in terms of QoS parameters (loss, delay, delay jitter, etc.), between the individual IntServ flows that are aggregated and corresponding DiffServ flow aggregate is given in [6]. A similar work [7] uses a newly proposed sender-initiated resource reservation mechanism, instead of RSVP, at the edges of the network. There are experimental results reported on commercial products as well. Performance analysis for such a case, using an industry implementation of RSVP on a small testbed of Cisco 4700 routers [8], indicates that the control message overhead can be carefully controlled without any serious negative side effects.In all the above works the implications of policy control and mobility support on router architectures are not considered. This paper addresses the issue of how QoS support can be extended for Mobile IP and managed through a Policy Based Network Management system.3The testbed architectureTo study the behaviour of the proposed router control plane in an actual network, we have set up a testbed that models the real IP network. The testbed network configuration is given in Figure 1 and is used as the common ground for the rest of this paper. Three domainsin the experimental testbed are shown in the figure – two access networks that support IntServ QoS, interconnected by one core network that supports DiffServ QoS. The traffic flow between the access and core networks is subject to Service Level Agreements (SLAs) - a contract of service level specifications (bandwidth, delay, etc.) - between two adjacent networks. Some of the functional requirements of a router depend on where it is used in the network; the required components in these routers vary accordingly (refer to section 4). The end user reaches network services through a wired or wireless access network. The different types of network elements shown in figure are described below.Figure 1. The QoS/PBN network architectureThe Access Router (AR) is the first router through which a Host is connected to the network. The Edge Router (ER) is part of the access network. The ER sends and receives network traffic between its own administrative domain (AD) and other domains. The ER is also responsible for inter-AD traffic management (classification, metering, marking, shaping) and will usually have SLA with other ADs. AR and ER will carry out per-flow handling of packets.The Bord er Router (BR) forwards traffic from the administrative domain of the core network to the outside world and also receives traffic from other domains. It also carries out inter-AD traffic management and will usually have SLA with other ADs including access networks. The Core Router (CR) performs only transport services such as packet forwarding with DiffServ QoS. BR and CR are only required to do aggregate traffic handling.The Directory Server (DS) stores various policies and information such as configuration and profiles about different network entities. Typically, every AD will have one directory server.The Policy Server (PS), also referred to as Policy Decision Point or PDP, is the evaluation unit for network usage, in compliance with policies stored in the directory server. The PS will usually have a policy management console for editing, viewing and changing policy entries. The policy related information is conveyed between thePDP and various network devices, also referred to as Policy Enforcement Points or PEPs, by using the Common Open Policy Service or COPS protocol [9]. The policies are conveyed between the PS and the DS using a directory access protocol such as the Lightweight Directory Access Protocol (LDAP) [10]. The communication protocol between different policy servers is beyond the scope of this project.In Figure 1 there are two Wireless LAN networks shown – in Domain 1 (AD1) and Domain 3 (AD3). With Mobile IP [4] support in place, the mobile node, for example laptop, can move between these two domains seamlessly without changing its IP address or discontinuing the sessions. The Home Agent (HA) for the mobile node is Access Router (AR) in AD3. The Foreign Agent(FA) is the Access Router (AR) in AD1.4The router control planeA router is a network device that receives packets from its interfaces and forwards them based on routing information maintained in its routing table. The packet forwarding mechanism includes selection of the output interface, selection of the next hop, encapsulation, etc. Once this is done, packets are queued on the respective output interfaces, from where they are scheduled to be sent out to the physical link. Traditionally, routers followed FIFO scheduling for best effort data delivery. More sophisticated scheduling mechanisms are used in new generation of routers, to achieve QoS. In the Internet, packets usually have to pass many routers before reaching their destination. The service offered by individual routers determines the overall end-to-end performance achieved in delivering each packet.Figure 2. The router control plane In Figure 2, the traditional router control plane has been extended to provide QoS, to support mobility and for centralized policy based configuration and control.Providing QoS requires differential treatment of traffic, generally referred to as traffic control, when it is queued for delivery at the output interface, and in some cases at the input interface. A framework to support QoS is created based on the Linux kernel traffic control elements. The Traffic Control (TC) block in Figure 2 represents this framework. The Traffic Control Adaptor (TCA) block implements a server in the user-space that serves the traffic control configuration requests by upper layers.The Routing Protocol Stack implements a routing protocol (e.g., OSPF, RIP) and updates the Routing Table (RT). The RT is referenced by the Packet Forwarding Block to decide where to send packets. The Mobile IP Stack adds IP mobility support to the system and is only required in routers that acts as Home Agent (HA) or Foreign Agent (FA).The COPS-PEP block establishes communication between the router and the Policy Server for policy decisions. Policy decision may be the result of a push of policy rules from the policy server or a result of an outsourced request by RSVP as explained in Section 4.4. These two models are supported through the sub-components COPS-PR and COPS-RSVP respectively.The following open source implementations of different protocols and components are used in the prototype implementation: COPS implementation from the Lulea University, RSVP implementation from the Technical University Darmstadt [11] and Dynamics Mobile IP implementation from HUT [12].In this work, the above-mentioned implementations were extended to create the complete integrated architecture shown in Figure 2, resulting in an environment that provides QoS guarantees and mobility subject to policies. More details on Traffic Control, QoS signaling and Policy Control are given in the following sections. These sections focus on the enhancements and modifications made to the above implementations to successfully integrate the system.Normally, when Internet traffic load increases, they would experience packet loss, longer delay for packet delivery and larger jitter when packets are received. The main reason is congestion at the routers. To guarantee QoS, where a level of assurance is given to the users on packet delivery, prioritization for packets from different users, groups, or applications is needed at these routers. In order to support this, traffic control mechanisms that can classify, police, mark and prioritize packets, are used. These mechanisms are normally applied to packets at their outgoing interface (egress traffic control), but can also be applied to incoming packets (ingress traffic control). Traffic control elements (classifiers or filters, meters, algorithmic droppers, queues and schedulers) are available in Linux kernel to create ingress and egress traffic control frameworks. More details on Linux traffic control can be found in [13].4.1Traffic controlFigure 3b. Traffic control framework at egress In this section, a traffic control framework that makes use of Linux traffic control mechanisms is described. This framework has been used in the experimental testbed for measurements. The framework is designed in a flexible manner such that it can be configured to support both the IntServ and DiffServ mechanisms in the same router. It is also possible to alter the configuration so that the router can act as AR, ER, BR or CR for its QoS related functionality. These configurations can be dynamically controlled through a policy server based on policy rules. See Figure 3a and Figure 3b for a high-level diagram of the framework.At the ingress interface where packets enter the router, metering of individual flows or groups can be done to limit their traffic based on pre-defined profiles and to perform certain actions (eg. drop or change of service) on it. The classifiers shown at the ingress of Figure 3a dispatch these flows or groups to different meters and policers. IntServ classifiers select packets based on source and destination IP addresses and port numbers. These classifiers correspond to different RSVP flows and are created by the RSVP layer (refer to Figure 2). DiffServ classifiers on the other hand select packets based on DSCP value in the IP header. There is no queuing of packets at ingress.As shown in Figure 3b, classifiers at the router egress select packets to different queues, from where these packets are sent out by the scheduler. The classifiers at egress behave similarly as the ingress classifiers mentioned above. IntServ classifiers select packets for RSVP queues and DiffServ classifiers for DiffServ queues. Separate queues are created for individual RSVP flows. Optionally metering and policing can also be done at the egress.DiffServ queues handle classes of traffic that are identified based on the DSCP value in the IP header. A separate queue is created for each standard DiffServ class. The traffic that is not selected by any of the RSVP or DiffServ classifiers goes to the Best Effort queue.Each of these queues at the egress can be specified to ‘mark’ packets, in which case the DSCP value of all packets queued to this queue will be (re)set to the specified value. The queued traffic can also be subjected to different buffer management schemes or shaping.The total bandwidth allocated for RSVP queues, DiffServ queues and the Best Effort queue is fixed for any given configuration. However the share for each of them can be varied. As an example, the experimental testbed setup contains no RSVP queues for a core router. The bandwidth share can be varied based on policy rules using the policy server.RSVP is used as the QoS signaling protocol in the testbed to exchange an application’s QoS requests to every RSVP supporting router in the sender-receiver path. Enhancements are made to standard RSVP implementation to support policy control, mobility and to interface to the traffic control block (refer to Figure 2). RSVP uses this interface to create or remove queues and classifiers.For policy control, an interface is defined between RSVP and the COPS-RSVP sub-component of COPS-PEP. RSVP uses this interface to outsource policy decisions to the PDP.The other enhancement integrates mobility with QoS. Researchers have proposed different mechanisms for providing QoS for mobile users. The focus of work here is limited to making a new RSVP based reservation for the user in a different subnet than his home network when the user switches his network. Here we assume that resources are available for the user to continue with the QoS session in the foreign network. The resource allocation for mobile users will be controlled through the policy based management system. The reservation mechanism is designed to set up the new reservation in the shortest possible time.RSVP relies on periodical RSVP messages to refresh reservation states. When a mobile node makes a handoff from one location to another, it takes some time for reservation states to be re-established along the new path.4.2QoS signaling and mobile IP supportThis delay is referred to as re-establishment delay. During this period, the ongoing flows will receive best-effort treatment without any QoS guarantee. In the worst case, the re-establishment delay is equal to the RSVP refresh interval. Mobile IP (MIP) and RSVP need to be integrated so that reservation states can be established along the new path immediately after handoff.The signaling involved in setting up a session between a mobile node and a correspondent host can be seen in Figure 1. The mobile node in figure is connected through the access router of domain 1. This domain is a foreign network for the mobile node and the access router through which it is connected now is the foreign agent. This node was originally part of domain 1, its home network, and the home agent is the access router from this network. The correspondent host is connected to one of the subnets of domain 3. The control plane architecture for a host that supports QoS and policy based management is essentially similar to that of a router shown in Figure 2. But for hosts, some of these components are optional. A mobile node may not need the COPS-PEP layer to make policy requests and also the traffic control layer. In addition, a correspondent host has no special support for mobility and does not require a Mobile IP Stack. Applications in hosts are running on top of these layers. When a mobile node makes a handoff, the MIP layer of this node notifies this event to RSVP layer of the same node. RSVP will immediately initiate a new reservation along the new path or RSVP refresh messages will trigger a new reservation from the sender in case this node is not the sender for those ongoing sessions. The access routers used as foreign agents also have MIP running along with RSVP and routing protocols. The core routers transparently forward signaling messages (i.e., MIP, RSVP, etc.) as normal (prioritized) IP packets.When the solution described above is used for ensuring QoS for mobile users, short interruptions can be expected during handover. This is due to the delay in setting up QoS for the new path. Enhancements have been proposed to reduce or to totally avoid this delay. There are many compromises to be taken in this regard. Work on a new mechanism to minimize QoS setup delay during handover is currently under progress in this project.Policy based management is supported at the routers through the COPS-PEP block in Figure 2. This block communicates with the PDP and handles policy enforcement. The sub-component COPS-PR supports provisioning and COPS-RSVP supports outsourcing. The policy provisioning and outsourcing mechanisms are briefly outlined in Section 4.4.The COPS-RSVP layer mainly acts as an agent for forwarding the policy decision requests from RSVP to policy server, and vice versa. The COPS-PR layer implements the handling of policy provisioning mechanism for the PEP. This layer interacts directly with the traffic control block to modify the traffic control framework to reflect the policy decisions received as part of provisioning.State information is stored in the COPS-PEP block corresponding to the installed policies. The state representation is defined in a standard format to keep it vendor independent. For a DiffServ router this information is in terms of Traffic Control Blocks (TCBs), which contain zero or more classifiers, meters, actions, algorithmic droppers, queues and schedulers. These elements are arranged according to the QoS policy being expressed.Enhancements were made to the original Lulea COPS implementation to support RSVP policy control, to interface COPS PEP to RSVP and traffic control blocks and to map the COPS PEP state information to corresponding Linux traffic control configuration.4.4Network operation, policy provisioningand outsourcingThe typical sequence of steps involved in invoking a QoS aware application in the policy-based network shown in Figure 1 is described below.When the system starts up, routers send configuration requests to the PS. PS fetches the relevant policies from the DS and these policies are then pushed on to the routers. This model is referred to as policy provisioning. The configuration of routers to be used as AR, ER, BR or CR and the basic resource allocation (bandwidth allocation for IntServ, best effort and DiffServ flows) for them is thus dynamically carried out.Per-flow policy decisions are made based on a policy outsourcing model. In this model, the application at the end-user’s host first requests network resources using a QoS signaling protocol, the signaling protocol in this case is RSVP and the request is sent as a PATH message. When the request is received, the AR queries the PS. The PS retrieves policy related data from the DS corresponding to the user and application, interprets the data and formulates the corresponding policy decision. Examples for policy related data are user profile, service level agreement and time-of-day restrictions. The decision is then sent back to the AR. If the resource request is not rejected, the request message will be forwarded to the next routers in the path to the receiver. The receiver, after receiving this request, will initiate a reservation along the path followed by the request message. A RESV message is sent for reservation in case of RSVP. The RSVP capable routers may again contact the policy server to verify the reservation request and check for resource availability locally before committing4.3Policy based managementthe resources in the router. If resources are available, a reservation is made for the requested resources at this router. If reservation is successful at all nodes along the data path, the application will start transferring its data using the network resources that are reserved for it. While the traffic flow is active, periodic QoS signaling messages may be exchanged across the network to refresh the reservation state. Once the application completes usage of network resources, the allocated resources will be released through explicit RSVP signaling or time-out.In addition to dynamic, application initiated resource reservations, resource management policies from time to time can be pushed to the routers by the policy server (provisioning), where these policies will be enforced to reserve resources. This will be useful if the network operator wants to allocate resources explicitly for a particular application and user without involving any QoS signaling.5Experimental resultsExperiments were carried out to study the implications in RSVP signaling performance for routers that are designed to support traffic control and uses policy based management. The results will help to assess the feasibility in deploying such networks. The experimental setup is shown in Figure 1. RSVP sessions were set up between two hosts, the Correspondent Host or CH and the Mobile Node or MN. One or two RSVP enabled routers were used between these hosts in the experiment. If AR1 is the access router in domain 1 and AR2 is the access router in domain 3, the test scenario can be represented as, [MN]Å-----Æ [AR1] Å----Æ [AR2] Å----Æ [CN]AR1 and AR2 are RSVP capable routers. Routers that do not support RSVP are not considered in this setup. The routers are Pentium-III, 700-800Mhz machines with 256MB of RDRAM running RedHat-7.3, kernel 2.4.5. All program modules were compiled using gcc version 2.95.3. Sender and receiver test clients were used to establish RSVP sessions. In a real setup, sessions could be started by users at random instances of time and each of these sessions may last for varying intervals of time. For the derived results to be useful to study the behavior of real Internet scenarios, session start-up times and session durations are varied based on random distributions. The sender sends out PATH messages at random time intervals according to a Poisson distribution. The sender also records the time elapsed until it receives the RESV message from the receiver, which is considered RSVP response time. After an exponentially distributed random lifetime, the RSVP session is torn down by PATH TEAR message from the sender. In this case, lifetime with a mean value of 1 second was chosen for the experiment. In order to generate higher loads, four sender-receiver pairs were used. The end to end delay for setting up of RSVP sessions is measured against different session rates by varying the session rate in steps. To make sure that the results are balanced, sessions were setup from CH to MN and from MN to CH.End-to-end RSVP response times for different session rates were taken for the following scenarios:1.Two RSVP enabled routers between end hosts; therouters do not invoke traffic control or use policy based management for outsourcing policy decisions.2.One RSVP enabled router between end hosts; therouter does not invoke traffic control or requests forpolicy decisions.3.No RSVP enabled routers between end hosts4.Two RSVP enabled routers between end hosts; therouters invoke traffic control without policy decisions.5.Two RSVP enabled routers between end hosts; therouters invoke traffic control based on policy decisions.6.One RSVP enabled router between end hosts and thisrouter invokes traffic control based on policy decisions.Figure 4. End-to-end RSVP signaling response time for different session ratesFigure 4 shows the end-to-end RSVP signaling delay for different scenarios. The diagram shows end-to-end delay dependent on the number of sessions per second. The delay increases significantly at higher session rates if policy based management is pared to the increase in delay due to traffic control configuration. The current implementation of the policy management system processes policy requests sequentially, resulting in higher delays for policy decision at higher session rates. By increasing parallelism in the PDP this can be improved. Also, it may not be required to make policy requests from R = Number of RSVP enabled routers between test clientsP = Policy Based Management applied for QoS requestsT = Traffic Control enabled in the routersevery router in a domain in most practical cases. The policy decision delay can further be minimized by caching some of the policy decisions in the routers itself. With this cached policies the routers can act as local policy decision points or LPDPs. This is particularly useful in mobile IP handoff scenarios. When a user moves to different networks, it may be required to make frequent policy decisions for the same user to ensure proper usage of resources. Concept of LPDP is considered for future work in this project.The delay for invoking traffic control is significant (~8ms at each router), but remains more or less constant against change in session rate (compare e.g. graphs [2-R] with [2-R, T]). The higher traffic control delay is mainly due to the fact that in the current design of traffic control framework more importance was given for flexibility than optimization. Optimizing this framework can significantly improve the end-to-end performance.Without traffic control and policy control, the end-to-end delay is very small and it does not increase significantly with higher load. According M. K arsten [11], KOM-RSVP can achieve a latency of 5 ~ 6 ms when load is 20000 sessions per second, which shows that even along a path with a large number of hops, the end-to-end delay is very likely to be acceptable.6ConclusionThis paper presents an architecture for the control plane of a software router that integrates signaling protocols and control mechanisms for QoS, policy based management, and mobility. The issue of how QoS support can be extended for Mobile IP and managed through a Policy Based Network Management system is described here. A flexible framework is created to support both IntServ and DiffServ QoS and this framework can be adopted in different types of routers. The architecture is implemented in a software router based on Linux and the experimental results are reported. The results demonstrate that the end-to-end delay in the network, where the nodes in access networks use the above architecture, is quite acceptable and scales very well with high signaling load. Normally, an access router is not required to support high traffic load, suggesting that a software router can suitably be used as an access router. Software router has advantages of flexibility and programmability.As for future work, the duration of handoff interruption on ongoing QoS session need to be measured quantitatively. It is a critical issue to minimize handoff interruption to the level that is acceptable to real-time streaming application like VoIP and VoD. A new framework for traffic control is developed that optimizes data handling. This framework needs to be integrated into the testbed and new performance measurements taken. Adding parallelism to the policy decision mechanism and improving the performance of the policy based management system is another task to be completed. AcknowledgementThe authors acknowledge the contributions of the members of the QoS/PBN project teams at the Institutefor Infocomm Research and Siemens ICM.References[1]Y. Bernet et al., “A Framework for Integrated Services Operation over Diffserv Networks”, IETF RFC 2998, November 2000.[2]R. Braden et al., “Resource ReSerVation Protocol (RSVP)- Version 1 Functional Specification”, IETF RFC 2205, September 1997.[3] A. Ponnappan et al., “A Policy Based End-to-end QoS Management System for the IntServ/DiffServ Based Internet”, Policy 2002 Workshop, June 2002.[4] C. Perkins, ed., "IP Mobility Support", IETF RFC 2002, Oct. 1996.[5]W. Almesberger et al., “A prototype implementation forthe IntServ operation over DiffServ networks”, Proceedings of IEEE Globecom 2000, Nov-Dec 2000.[6]T. Chahed et al., “On mapping of QoS between integrated services and differentiated services”, Eighth International Workshop on Quality of Service, June 2000. IWQOS. 2000.[7]G. Zhang, H.T. Mouftah, “End-to-end QoS guarantees over Diffserv networks”, Proceedings of Sixth IEEE Symposium on Computers and Communications, July 2001.[8] A. Neogi, T. Chiueh, P. Stirpe, “Performance analysis ofan RSVP-capable router”, IEEE Network, Sept-Oct 1999.[9] D. Durham, ed. et al, "The COPS (Common Open Policy Service) Protocol", IETF RFC 2748, Jan. 2000.[10]W. Yeong et al., “Lightweight Directory Access Protocol“, IETF RFC 1777, Mar. 1995.[11]M. K arsten et al., ``Implementation and Evaluation of KOM RSVP Engine‘‘, Proceedings of IEEE Infocom 2001.[12]The Dynamics - HUT Mobile IP system, developed at Helsinki University of Technology (HUT), http://www.cs.hut.fi/Research/Dynamics.[13]Almesberger, Werner. Linux Traffic Control – Implementation Overview, Technical Report SSC/1998/037, EPFL, November 1998.。