Inter-Domain QoS Provisioning and Accounting

Tomoichi Ebata (ebata@sdl.hitachi.co.jp)
TAKIHIRO, Masatoshi (takihiro@sdl.hitachi.co.jp)
Shigeru Miyake (yake@sdl.hitachi.co.jp)
Minoru Koizumi (m-koizu@sdl.hitachi.co.jp)

HITACHI, LTD. Japan

Felix Hartanto (hartanto@fokus.gmd.de)
Georg Carle (carle@fokus.gmd.de)

GMD FOKUS Germany

QoS[QOS]-capable communication network devices are now available, such as IP routers and switches, thus enabling the implementation of QoS-guaranteed services. Policy based network management (PBNM) is a framework to control these network devices with management rules that are called "Policy". At present, most network engineers on PBNM cope with only intra-domain QoS-guaranteed services, but an enhanced framework for inter-domain QoS-guaranteed services required for real commercial networks that communicate with networks managed with different policy. On real commercial networks, service providers have to charge customers for the inter-domain QoS-guaranteed services.
We focus on inter-domain QoS provisioning and accounting. First of all, we discuss the policy framework and its architecture. Based on this framework and architecture, we propose two protocols: a "Policy Advertisement Protocol (PAP)" that distributes local policies among domains, and a "Policy Negotiation and Notification Protocol (PNP)" that sets up a QoS-guaranteed communication path among domains[INTD].
Our experimental network confirms that this inter-domain QoS provisioning is effective when service providers provide inter-domain QoS services from the point of view of network administrator's tasks, performances, and network traffic.

1. Introduction

QoS-guaranteed services can now be provided because of the development of communication network devices, for example QoS-capable routers. The Internet Engineering Task Force (IETF) has started to examine "policy", which is sets of rules, as a means of managing and controlling network devices and providing services.

In addition, the use of Service Level Agreements (SLAs)[SLA] has been introduced for the benefit of Internet Service providers (ISPs) and their users. These SLAs are contracts for services that ISPs provide their users. They contain information on QoS-guaranteed services and the accounting cost of the services. Both QoS-guaranteed services and the accounting are critical to the SLA as far as Internet services are concerned. Policy Servers [TERM] (PSs) that uniformly manage the above QoS-guaranteed services, accounting, and SLAs are being developed now. This management method in which PSs control the network using policy is called "Policy Based Network Management (PBNM)".

At the same time, inter-domain QoS-guaranteed services have become important, leading to a need for contracts that refer to different domains, e.g., different ISP networks, enterprise networks, and carrier networks. In the case of inter-domain environment, as each domain is managed in accordance with a specific policy, it is natural that a policy in one domain is different from a policy in another domain. Therefore, it is impossible to seamlessly provide inter-domain QoS-guaranteed services and accounting within the current PBNM framework. The policy working group (WG) in the IETF has started to examine policy core schemas and QoS schemas that are currently being used in Policy Servers. But the schemas are supposed to be used within intra-domain networks.

The purpose of this paper is to show how to implement inter-domain QoS-guaranteed services and inter-domain accounting. To implement inter-domain QoS-guaranteed services and inter-domain accounting, network service providers, for example ISPs, must provide users with an inter-domain QoS provisioning service and an inter-domain accounting service as fundamental services.

At the very least, it must be possible to exchange policy among domains because one domain must know the policy of another domain. This paper focuses on the inter-domain policy framework needed to implement this policy exchange process, the specific contents of policy and their properties, and network architectures suitable for policy exchanges. It also describes two protocols, the "Policy Advertisement Protocol (PAP)" and the "Policy Negotiation and Notification Protocol (PNP)" that can be used for this implementation.

This paper is organized as follows.
Section 2 presents the inter-domain policy framework and its application to inter-domain QoS provisioning and inter-domain accounting. This section also presents ten different kinds of policy. Six of these are identified as relevant for inter-domain QoS provisioning and accounting processes. At the end of this section, we explain why and how these policies are exchanged.
Section 3 presents two architectures for policy exchange. One architecture uses a so-called super PS, while the other does not. The utility of these architectures are compared, and some problems with them are discussed.
In Sections 4 and 5, the message formats and further details on the two protocols are explained.
Finally, Section 6 presents the results and evaluation of an experimental network that has PSs that use the two protocols.

2. Inter-domain policy framework

2.1 Inter-domain services

In this paper, the inter-domain services are composed of three kinds of services. Fig. 2-1 shows building blocks of inter-domain services.

General services

services that customers use among domains, e.g., IP telephony, TV conference. But this paper does not give the details on this.

Elemental services

services that provide QoS provisioning and accounting for the general services among domains. This services are composed of policies that are defined in Section 2.2

Network service

services that provide QoS-guaranteed network. This services are composed of QoS-capable communication network devices, e.g., QoS-guaranteed routers and IP meters. But this paper does not give the details on this.

2.2 Definition of policy

In this paper, policies have attributes and rules that are composed of the attributes. They are similar to be objects of the object-oriented technique.

When events as conditions are inputted to the policy, the policy evaluate the events in accordance with the rules, and output events as actions. The events contain request or response information.

2.3 Inter-domain policy framework

The inter-domain policy framework is shown in the figure 2-3.

Customer-specific policy, service-specific policy and resource-specific policy are to be exchanged with other domains. The main focus of this paper is the inter-domain exchange of these policies.

Network-function-specific policy and network-element-specific policy are used for controlling the network devices for intra-domain QoS provisioning and accounting.

The customer-specific policy controls the customer access to the available services within a provider's domain. The policy is compose of attributes of a customer (e.g., priorities, usable services, valid time, usable resources) and rules (e.g., "If Mr. A has gold priority, then he can use all services."). There are two kinds of customer-specific policy, "Customer Policy" and "Billing Policy".

The service-specific policy controls how a service should be provided to a customer. The policy is composed of attributes of service (e.g., expected QoS, valid time, cost) and rules (e.g., "If service A is available, then its cost is one dollar a minute."). There are two kinds of service-specific policy, "Service Policy" and "Charging Policy".

The resource-specific policy controls how actual network resources are available. This policy is composed of attributes of resources (e.g., network medium, bandwidth, delay, jitter, Maximum Transmission Unit (MTU)), status information (kinds of service, path, and time), and rules (e.g., "If there is not enough bandwidth for Mr. A's service, reduce other customer's bandwidth"). There are two kinds of resource-specific policy, "Flow Policy" and "Accounting Policy".

The network-function-specific policy make final decisions for each network devices, or collect network status information from them. This policy is composed of attributes of network functions and rules (e.g., "If the utilization rate of link A is above 90%, use link B"). There are two kinds of network-function-specific policy, "Policing & Routing Policy" and "Reading Policy".

The network-elements policy defines the configuration of network elements for providing a network function associated with a service. The policy is composed of attributes of network devices (e.g., setting parameters and their values) and rules (e.g., "If bandwidth is available, then set bandwidth parameter enable, and mark packets as green"). The policy is also composed of attributes of flow (e.g., real-time measurement values of bandwidth, delay) and rules (e.g., "If "user - IP address" is assigned, relate flow with IP address"). There are two kinds of network-elements policy, "Classifying & Queuing Policy" and "Metering Policy".

2.3.1 Inter-domain QoS provisioning policy framework

Figure 2-4 shows the inter-domain QoS provisioning policy framework, which is applied to the inter-domain policy framework shown in Fig. 2-3. The QoS provisioning process starts with the requests from QoS-guaranteed services. This process is executed left-right and/or top-down.

The QoS-guaranteed services output the following information: (1) which service is needed,(2) who uses the service, and (3) how to use the service, in each of the Customer policy, the Service policy, and the Flow policy.

For example, when a QoS-guaranteed service puts out some requests, the customer policy checks the attributes of a customer and rules, and outputs the service request to the service policy. After checking the service attributes and rules, the service policy outputs the flow requests to the flow policy. With the same process, the flow policy outputs the network function requests and the policing and routing policy dictates the classifying and queuing policy to be used for configuring the communication devices.

2.3.2 Inter-domain accounting policy framework

Figure 2-5 shows the inter-domain accounting policy framework that is applied to the inter-domain policy framework shown in Fig. 2-3. The accounting process starts with measurement for a specific network packet. This process is executed bottom-up and/or right-left .

For example, each communication devices keeps observing network packets. The metering policy that are in a communication device checks the packet attributes and outputs the metering response to the reading policy. After checking the metering response, the reading policy outputs the reading response to the accounting policy and/or the charging policy and/or the billing policy. With the same processes, the accounting policy outputs the accounting response and the charging policy outputs the charging response. Finally, the billing policy outputs the billing response in QoS-guaranteed services.

Thus, these responses are outputted as the information of (1)the customer's bill, (2) the status of the QoS-guaranteed service, and (3) the status of resource utilization in the QoS-guaranteed services.

2.4 Exchanged policy

To execute inter-domain QoS provisioning and accounting, a policy exchange is needed for the following reasons.

(1) To provide inter-domain services, it is necessary for one domain to know

(a) which services are available in another domain

(b) which customers exist in another domain

(c) how many resources are available in another domain

(2) To provide inter-domain accounting services, it is necessary to have information on (a), (b), and (c), and

(d) additional information such as charging tables for services and resources in another domain.

It is also necessary to notify customers of the costs for used services and resources.

Thus, the policies of each domain involved in end-to-end service provisioning make it possible to provide inter-domain QoS provisioning and accounting.

The above policies are exchanged in different ways because of the different property of each policy. Three ways of exchanging policies are now introduced.

2.4.1 Advertised policy

This policy must be possessed by all relevant domains. So this policy may be carried by broadcast communication.

1. Inter-domain QoS provisioning policy framework

(a) Customer policy

(b) Service policy

(c) Flow policy

2. Inter-domain accounting policy framework

(a) Charging policy

2.4.2 Notified policy

(1) Request and response

This policy are used as a trigger to start inter-domain QoS provisioning. This policy may be exchanged by request and response communication as shown in Figure 2-7(a). A domain that sends requests is called a "Source Domain" and a domain that sends responses is called a "Destination Domain".

1. Inter-domain QoS provisioning policy framework

(a) Customer policy

(b) Service policy

(c) Flow policy

(2) Notification

This policy is used as accounting state reports to notify one domain of the cost of inter-domain QoS-guaranteed service. This policy may be notified by unicast communication, in Figure 2-7(b). A domain that sends notifications is called a "Source Domain". A domain that receives them is called a "Destination Domain".

1. Inter-domain accounting policy framework

(a) Billing policy

(b) Charging policy

(c) Accounting policy

3. Inter-domain policy exchange architecture

In this section, the architecture of the inter-domain policy exchange is now explained.

3.1 Two types of architecture

Figure 3-1 shows two architectures.

In architecture (a), policies are exchanged through an Super Policy Server (SPS).
A direct policy exchange process among domains is unnecessary here and the load on PS becomes low. Because all PSs that execute QoS provisioning and accounting connect with the SPS directly, scalability and flexibility are sacrificed. And no one knows who manages the SPS.

In architecture (b), on the other hand, the load on PS becomes high, but scalability and flexibility problems are not as serious as in (a). Network administrators manage their intra-domain only with the current Internet architecture.

Architecture (b) is now explained as follows.

3.2 Subject

The subjects of architecture of type (b) are showed.

(1)Different exchange means for different properties of policy

In Section 2, it became clear that the three communications "advertisement", "request and response", and "notification" are needed. So PSs need to have such means of communication.

(2)Low-load policy exchange process

Because the six kinds of policy are needed to be shared among domains, the load on PSs may unfortunately be high. If the exchange processes are executed at the same time, the processing power of PS may become lower. So PS should not exchange both the advertised policy and the notified policy at the same time.

3.3 Outline of policy exchange protocol

In order to resolve above the subjects, PSs should implement two phases of policy exchange processes.

(Step.1) Policy advertisement phase

In this phase, "advertisement" is executed in order to advertise policies among domains.

(Step.2) Policy request-response and/or notification phase

In this phase, "request and response" and "notification" are executed. After PSs refer to the advertised policy, PSs send and/or receive the policy that contains request and/or response contents. PSs send and/or receive the policies that contain the notification about the services too.

The "policy advertisement protocol" is used to advertise the policy. To support this functionality, BGP4 [RFC1364][BGP4] is available. BGP4 is able to carry not only routing information but also additional information in a BGP4 message. In this paper, the enhanced BGP4 is proposed to advertise the policy of each domain.

The "policy negotiation and notification protocol" is used to negotiate and notify domains of the policy. To support this functionality, COPS (Common Open Policy Service)[COPS], which is currently being standardized by IETF, is available. COPS is a signaling protocol, and it exchanges policies between a policy decision point (e.g., policy server) and the policy enforcement point (e.g., router). This protocol is able to convey some policy and signal to execute some actions corresponding with its policy between PDPs (Policy Decision Points, like PSs) and PEPs (Policy Enforcement Points, like routers). But COPS seem to be used within intra-domain networks. In this paper, the enhanced COPS is proposed to negotiate and notify domain of the policy between PDPs (for example, PSs).

3.4 Preparation

The premise system structure and the preparation before starting the system is explained. Figure 3-2 shows a sample system structure.

Each PS (PSa, PSb, PSc, PSd) owns the policies of each domain. There are some routers and other devices in each domain. Border routers (BRs) in a domain are connected with other BRs in other domains. The PSs manage some devices in the domain and control them (for example, intra-domain routing) according to the policy. And PSs exchange the policy among domains and execute QoS provisioning and accounting.

Network administrators must get the following information from neighboring domains, which are connected by physical communication lines, before PSs start the policy exchange process and inter-domain QoS provisioning and accounting.

• Domain ID (in short AS-ID) and IP address for PS (in short PS-IP)
• IP address of BRs (in short BR-IP) of inside and outside interface
• Resource information about the inter-domain physical communication lines (which are called inter-domain links, in short I-D links) (e.g., network medium, maximum bandwidth, range of packet loss, minimum delay, minimum jitter, MTU)

Next, Network administrators obtain the following information and store it in the PS's storage device. Part of this information is treated as the policies.

• Customer information (for example, host IP, priority, available service name, right of the inter-domain service use)
• AS-IDs that the above customer are allowed for inter-domain communication.
• Inter-domain link ID that shows that the above customer's host can arrive at the domain of the above AS-ID

In the next section, it is assumed that PSs already have the above information.

4. Inter-domain policy advertisement protocol

In this section, the inter-domain policy advertisement protocol (in short PAP), which is an enhanced BGP4 protocol, is explained.

4.1 Outline

There are some hosts and networks that are specified by the addresses in one domain. The advertised policy is treated as a set of this information, and is distributed in another domain. A simple case is illustrated in Fig. 4-1.

The advertised policy in PSa of domain A contains at least the following information.

• Host a1 (Ha1) is in the domain A
• To execute services using Ha1, three inter-domain links (La1, La2, La3) are available that have QoS information (bandwidth, delay etc.).

This information of the policies is made by PSs that know about the structure of the domain network. For example, PSs make the flow policy of Ha1 with resource information (e.g., network medium, bandwidth) to Ha1 through La1.The policy is advertised by PAP.

Figure 4-1 shows how one policy in domain A is being advertised from PSa (Source PS) to PSc and PSd (Destination PS).

This policy is advertised with routing information on BGP4. A different process is that the advertised policy, the flow policy for example, may be transformed according to each PS in accordance with the network structure and/or state.

Firstly, PSa advertises its policy to PSc. The policy contains the following information.
If a customer in domain C would like to communicate with Ha1's customer in domain A, that customer can use the 3.5-Mbit/sec bandwidth through BRc1 in domain C. The bandwidth is limited by the minimum bandwidth through the whole path between Domain A and Domain C. The available bandwidth of each link is 3.5 Mbit/sec between BRc1 and BRa2, and 10.0 Mbit/sec between BRa2 and Ha1.

Secondly, PSc forwards its policy to PSd. This policy contains the following information.
If a customer in domain D would like to communicate with Ha1's customer in domain A, the customer can use the 2.7-Mbit/sec bandwidth through BRd1 in domain D. The bandwidth is limited by the minimum bandwidth through the whole path between Domain C and Domain D. The available bandwidth of each link is 4.0 Mbit/sec between BRd1 and BRc2 and 2.7 Mbit/sec between BRc2 and BRc1.

In Fig. 4-1, the bandwidth that is included in the flow policy is explained. But there are other attributes in the flow policy, which are transformed or forwarded by each PS as follows.

Attributes	Criterion
Bandwidth	Minimum value
Packet loss	Range value
Latency	Total sum value
MTU	Minimum value
Jitter	Total sum value

Thus, PS is allowed to transform the policies in this process, but this paper does not give the details on this.

4.2 Protocol

All PAP is based on BGP4, except for the policy transformation process.

4.3 Message format

The policies are advertised using UPDATE messages on BGP4. The format of a PAP message is shown in Figure 4-2.

5. Inter-domain policy negotiation and notification protocol

This section explains the policy negotiation and notification protocol (in short PNP), which is an enhanced COPS protocol, used among domains.

5.1 Outline

The PNP is outlined and the sample policy and functions are explained.

The PNP is used to convey some policy and to execute some actions according to its policy.
To control the use of these services among domains, a signaling function that triggers the control is needed to carry the necessary policy. For this PNP process, COPS (Common Open Policy Service) can be used. But COPS is only used between policy decision points like PSs and policy enforcement points like routers at present. Since the PNP must be enhanced in order to be able to adapt it for use between multiple PSs, PSs must function as both PDP and PEP.

Firstly, each PS makes the communication connection for the PNP. The connection process is that a PS sends a Client-Open (CO) message to another PS, and the PS that sent the CO receives a Client-Accept (CA) message from the another PS that sent a CA.

When a starting request from a QoS-guaranteed service occurs in the source domain (Domain A), the source PS (PSa) investigates several things. One is whether QoS provisioning is available or not in the domain. After investigating, the source PS (PSa) sends a Decision (DEC(Install)) message that includes a decision policy to the destination PS (PSc).

Next, the destination PS (PSc) investigates whether QoS provisioning is available or not in the domain, and sends a Report State (RPT (Installed/Not Installed)) message that includes its reply (accept negotiation or not) to the source PS (PSa).

But as Figure 5-1 shows, there is no direct physical connection between Domain A and Domain D. In this case, both PSa and PSd need to make a logical connection through Domain C. Thus, after PSc receives the Decision (DEC (Install)) message from PSa, PSc searches for a path that can connect Domain A with Domain D, and then investigates several items about the path. After that, PSc sends another Decision (DEC (Install)) message that includes the decision policy to PSd. PSd investigates whether QoS provisioning is available or not in the domain, and sends a Report State (RPT (Installed/Not Install)) message that includes its reply (accept negotiation or not) to PSc. After PSc receives PSd's reply, PSc sends a State (RPT(Installed/Not Install)) message to PSa.

Thus, this process is available regardless of the numbers of domain.

5.2 Protocol

All PNP is based on COPS, except for communicating between PDPs (PSs).

5.3 Message format

All message formats are based on COPS.

6. Evaluation

In this section, the results of an experimental network that has policy servers that use the above two protocols are presented. The experimental network was used to confirm that the inter-domain QoS provisioning process is effective when service providers provide inter-domain QoS services from the point of view of performances, network traffic, network administrator's tasks.

Figure 6-1 shows the experimental environment. Eight Network administrators operate PSa and/or PSb. And each network administrator does the same experiment four times.

6.1 Evaluation of PAP

(1)Purpose

The purpose of this experiment is to evaluate the effectiveness of PAP process from the point of view of (a)performances, (b)network traffic, and (c)network administrator's tasks.

(2)Contents of experiments

1. The time taken to update the advertised policy in PS in terms of
   1. transmission between PSa and PSb
   2. operation of network administrator on PSa
2. The traffic on network to update the advertised policy between PSa and PSb
3. Questionnaires were also sent to network administrators. The questions focussed on
   1. whether it was easy or not to input the advertised policy to the PS
   2. whether it was easy or not to modify the advertised policy
   3. whether it was easy or not to start up PAP process.
   4. whether starting up PAP process was fast or not

(3)Results and evaluations

1. The transmission time is 296.91 milliseconds on average. This shows PAP performs well in transmitting the advertised policy.
2. The operation time is 71 seconds on average an average.This shows that network administrators do not need more time to manage the policy advertisement.
3. The traffic is 1109.2 bytes on average. This shows that there was not so much traffic in updating the advertised policy.
4. The main information taken from the questionnaires was that network administrators felt it was hard to input the advertised policy, but they felt the policy was easy to modify. All network administrators felt that starting up and managing PAP process was simple and fast.

6.2 Evaluation of PNP

(1)Purpose

The purpose of this experiment is to evaluate the effectiveness of PNP process from the point of view of (a)performances, (b)network traffic, and (c)network administrator's tasks.

(2)Contents of experiments

1. The time required for PNP process to succeed in terms of
   1. operation for registration of the notified policy
   2. operation for cancellation of the notified policy
2. The traffic on network to convey the notified policy between PSa and PSb
3. Questionnaires were also sent to network administrators. The questions focussed on
   1. whether inputting registration of the notified policy was easy or not
   2. whether PNP process with registration is fast or not
   3. whether inputting cancellation of the notified policy was easy or not
   4. whether PNP process with cancellation is fast or not

(3)Results and evaluations

1. The operations for registration and cancellation are 15.0 seconds and 14.0 seconds on average.This shows PAP performs well in transmitting the advertised policy.
2. The traffic for operation of registration and cancellation are 1316.0 bytes and 904.0 bytes on average.This shows that there is not so much traffic during PNP process, which should mean no problems for real network management.
3. The main information taken from the questionnaires was that network administrators felt it was easy to manage the registration and/or the cancellation of the advertised policy. However, they felt the cancellation process was slow.

The conclusion drawn from these two types of experimental results is that this inter-domain QoS provisioning is effective when service providers provide inter-domain QoS services.

7. Conclusion

This paper has looked at how to execute QoS-guaranteed provisioning services and accounting services that are offered by QoS-guaranteed services among domains. results as follows.

(1) Inter-domain policy framework

The paper has presented the inter-domain policy framework and its application in terms of inter-domain QoS provisioning and inter-domain accounting. It has also presented ten different policies. Six of these are identified as relevant for inter-domain QoS provisioning and accounting control. These polices are exchanged by "advertisement", "Request and response" and "notification" communications.

(2) Inter-domain policy exchange architecture

This paper has investigated architectures for policy exchange, and proposes two architectures. The architecture without SPS is better in terms of scalability, flexibility and legacy of the current internet architectures.

(3) Inter-domain policy advertisement protocol

We proposed an inter-domain policy advertisement protocol (in short PAP), which is an enhanced BGP4 protocol, for the inter-domain policy advertisement.

(4) Inter-domain policy negotiation and notification protocol

We proposed a policy negotiation and notification protocol (in short PNP), which is an enhanced COPS protocol, for the inter-domain policy negotiation and notification.

(5) Evaluation

We ran experiments to confirm the performance of inter-domain QoS provisioning process using the two protocols. The conclusion drawn from these experimental results is that this inter-domain QoS provisioning is effective when service providers provide inter-domain QoS services.

8. References

[INTD]

T.Ebata, M.Takihiro, S.Miyake, M.Koizumi, F.Hartanto, G.Carle," Inter-Domain QoS Provisioning and Accounting", Internet-Draft, draft-ebata-inter-domain-qos-acct-00.txt, November 1999.

[QOS]

S. Gai, J. Strassner, D. Durham, S. Herzog, H. Mahon,F. Reichmeyer, "QoS Policy Framework Architecture",Internet-Draft, draft-sgai-policy-framework-00.txt, February 1999.

[SLA]

J. Strassner, E. Ellesson, B. Moore, "Policy Framework LDAP Core Schema",Internet-Draft, draft-ietf-policy-core-schema-04.txt, June 1999

[TERM]

J.Strassner,E.Ellesson,"Terminology for describing network policy and services",Internet draft,draft-strassner-policy-terms-01.txt, February 1999

[RFC1364]

K. Varadhan, "BGP OSPF Interaction", RFC1364, September 1992

[BGP4]

Y. Rekhter, T. Li, "A Border Gateway Protocol 4 (BGP-4)", Internet-Draft, draft-ietf-idr-bgp4-09.txt, September 1999.

[COPS]

J. Boyle, R. Cohen, D. Durham, S. Herzog, R. Rajan, A.Sastry,"The COPS (Common Open Policy Service) Protocol"Internet-Draft, draft-ietf-rap-cops-06.txt, February 1999.