CONCURRENCY AND COMPUTATION:PRACTICE AND EXPERIENCEConcurrency Computat.:Pract.Exper.2005;17:117–141Published online in Wiley InterScience().DOI:10.1002/cpe.931Performance technology forparallel and distributedcomponent softwareA.Malony1,∗,†,S.Shende1,N.Trebon1,J.Ray2,R.Armstrong2,C.Rasmussen3and M.Sottile31Department of Computer and Information Science,University of Oregon,Eugene,OR97403,U.S.A.2Sandia National Laboratory,Livermore,CA94551,U.S.A.3Los Alamos National Laboratory,Advanced Computing Laboratory,Los Alamos,NM87545,U.S.A.SUMMARYThis work targets the emerging use of software component technology for high-performance scientific parallel and distributed computing.While component software engineering will benefit the construction of complex science applications,its use presents several challenges to performance measurement,analysis,and optimization.The performance of a component application depends on the interaction(possibly nonlinear) of the composed component set.Furthermore,a component is a‘binary unit of composition’and the only information users have is the interface the component provides to the outside world.A performance engineering methodology and development approach is presented to address evaluation and optimization issues in high-performance component environments.We describe a prototype implementation of a performance measurement infrastructure for the Common Component Architecture(CCA)system.A case study demonstrating the use of this technology for integrated measurement,monitoring,and optimization in CCA component-based applications is given.Copyright c 2005John Wiley&Sons,Ltd.KEY WORDS:component software;performance;parallel;distributed;optimization1.INTRODUCTIONThe power of abstraction has played a key role throughout the history of scientific computing in managing the growing complexity of scientific problem solving.While the evolution of software abstractions and the technology that supports them has helped to address the challenges of scientific∗Correspondence to:A.Malony,Department of Computer and Information Science,University of Oregon,Eugene,OR97403, U.S.A.†E-mail:malony@Contract/grant sponsor:U.S.Department of Energy;contract/grant numbers:DF-F603-01ER25501and DE-F602-03ER25561Copyright c 2005John Wiley&Sons,Ltd.Received2December2003 Revised16February2004 Accepted25March2004118 A.MALONY ET AL.application development,it has at times been in conflict with the ability to achieve high performance. On the one hand,abstraction in software systems further distance the scientific application developer from the range of sources of performance behavior and possible performance problems.On the other hand,support for performance observation and analysis has been poorly integrated in software systems, making performance evaluation a more difficult task.As both the power and the complexity of scientific software environments and computing systems mutually advance,it is imperative that technology for performance evaluation and engineering keep pace and become part of an integrated software and systems solution.The software challenges of building large-scale,complex scientific applications are beginning to be addressed by the use of component software technologies(see,e.g.,[1–3]).The software engineering of scientific applications from components that can be‘plugged’together will greatly facilitate construction of coupled simulations and improve their cross-platform portability.However, the success of scientific component software will depend in the end on the ability to deliver high-performance solutions.Scientific components are more complex and diverse than typical software components or libraries,in their scale,execution modes,programming styles and languages, and system targets.Performance technology that lacks robustness,portability,andflexibility will inevitably prove incapable of addressing the software and platform integration requirements required for performance observation and analysis.Intra-component performance engineering addresses problems of individual component performance characterization,analysis,modeling,and adaptation. Inter-component performance engineering provides local and overall awareness of application performance and facilities to access that performance information for the application to utilize. Most importantly,performance engineering technology should be compatible with the component engineering methodologies and frameworks used to develop applications,or it will be neither routinely nor effectively applied by component and application developers.Our research work on performance technology for component software is defined by three objectives.Thefirst is a methodological and operation model for intra-and inter-component performance engineering.Here,we define how the performance engineering technology will integrate with the component architectures and frameworks being considered for scientific computing. The Common Component Architecture(CCA)Forum[4]is specifying component software extensions and infrastructure to address problems of parallel component interfaces,scientific data exchange, and cross-language interoperability.We chose the CCA specification as our reference archetype. The second objective is the development of technology to implement the methods and techniques required for intra-and inter-component performance engineering in the context of existing scientific component efforts.Our target audience for the performance engineering technology we are creating are the framework and application developers using the CCA specification.Specifically,we are integrating our T AU performance system[5]with the CCA software,such as the Scientific Interface Definition Language(SIDL)[6],the Babel component interface toolkit[7],and the CCAFFEINE framework[1]. Ourfinal objective is the application of the model and technology for performance engineering to real component-based scientific computing environments.The goal here is to demonstrate both the capability and utility of our performance engineering ideas.The three objectives above are covered in this paper.Section2discusses the use of component technology for scientific computing and motivates the general requirements for performance engineering.The functional operation of the CCA specification is also described.Our conceptual model for performance engineering of component software is presented in Section3.Here we describe Copyright c 2005John Wiley&Sons,Ltd.Concurrency Computat.:Pract.Exper.2005;17:117–141PERFORMANCE TECHNOLOGY FOR COMPONENT SOFTWARE119 a high-level methodology to technology development for intra-and inter-component performance engineering.Section4then considers the implementation of this technology in the context of the CCA software environment.Section5presents an application of our work with an emphasis on CCA performance modeling and optimization.Here we demonstrate the use of a CCA performance interface in measurement experiments to construct empirical performance models.We also show how, when the application runs,an optimizing component utilizes the performance application programmer interface(API)to gather statistics about the running application and decides which of the sets of similar components to choose for optimal performance.Our work has been targeted primarily towards high-performance computing(HPC)environments. However,it naturally extends to Grid computing both in terms of application of component technologies and the requirements for performance engineering.We briefly discuss the extensions and the issues that arise as part of the conclusions Section6.PONENT TECHNOLOGY FOR SCIENTIFIC COMPUTINGComponent technology extends the benefits of scripting systems and object-oriented design to support reuse and interoperability of component software,transparent of language and location[8].A component is a software object that implements certain functionality and has a well-defined interface that conforms to a component architecture defining rules for how components link and work together. The term component framework is used to designate a specific component architecture implementation. Component technology offers many advantages to scientific computing since it allows domain-level knowledge to be encapsulated in component building blocks that can easily,hopefully efficiently,be used in application development,removed from the concerns of how the component was developed or where it resides.As a result,scientists can focus their attention to overall application design and integration.Unfortunately,the three most widely-used component standards(CORBA[9],COM/DCOM[10], Java Beans[11])are ill-suited to handle high-performance scientific computing due to a lack of support for efficient parallel communication,insufficient scientific data abstractions(plex numbers), and/or limited language interoperability[12].Furthermore,often the software does not run on the systems scientists use,or simply runs too slow for their applications.ATo overcome some of the limitations of standard component software approaches for scientific computing,the CCA Forum[4]was started in1997to define the standard foundations of scientific component architecture and to facilitate the adoption of CCA tools and technologies.In addition,the U.S.Department of Energy(DOE)established the Center for Component Technology for Terascale Simulation Software(CCTTSS)[13]for purposes of developing the CCA software infrastructure and demonstrating its use for complex scientific simulations.Component programming,much like object-oriented programming,provides a model for constructing software such that units of code(components and objects)expose a‘public’interface to the outside while hiding their internal implementation ponents extend the object model by allowing components to dynamically discover and expose interface information,something that is Copyright c 2005John Wiley&Sons,Ltd.Concurrency Computat.:Pract.Exper.2005;17:117–141120 A.MALONY ETAL.Figure1.Two CCA components:One uses a‘P’port provided by the other.statically determined at compilation time in most object-oriented languages.Fundamentally,the CCA is a specification of the component programming pattern and the interface the components see to the underlying support substrate,or framework.The CCA allows components to describe their interfaces in the Scientific Interface Definition Languaged(SIDL)[6].Like the IDL used by CORBA,the interfaces are defined in a language independent manner and are not bound to the source code or compiled binary of a component.The IDL simply describes the public interface so that external parties can discover what services are available and how they must be called.In the CCA,a component is defined as a collection of ports,where each port represents a set of functions that are publicly available.A port is described using SIDL,and some form of wrapper exists in the implementation to map the SIDL interface to that of the implementation language.From the point of view of a component,there are two types of ports.Those that are implemented by a component are known as provides ports,and other components may connect to and use them.Other ports that a component will expect to be connected to and call are known as uses es and provides ports are connected together as shown in Figure1.The act of connecting components is referred to as component composition.When a component is instantiated and allowed to execute,it registers the provides and uses ports with the underlying framework.This information allows external components to discover what ports or interfaces are available,and ensures that expected relationships between components are fulfilled, before allowing execution.Port discovery is a service provided by the framework and is actually just another port that a component can connect to.For instance,a component can obtain a list from the framework of all components providing a specific interface or port.The component could then connect to each of the ports in the list in an iterative fashion and make calls on the methods within the connected port.Iterative access to a common set of interfaces presupposes that such a set exists. One of the primary benefits of component-based programming is the adoption of common interfaces for domain-specific purposes.This allows different teams of individuals to develop components based on this‘standard’component API.This,in turn,allows users of these components to pick and choose the particular component that bestfits their needs.The CCA architecture is the operational foundation for the implementation of CCA-compliant component frameworks,but it does not by itself directly specify how requirements of high-performance component systems are met.Rather,the intent of the CCA specification is to allow the preservation of performance.This takes four primary forms.First,the performance of the functional‘core’of a component(e.g.linear equation solver)should not suffer because it is implemented within a component.In general,this demands support for all core languages(e.g.C,C++,and Fortran77, 90and95),and support for all forms of parallel execution that the core software may employ. Copyright c 2005John Wiley&Sons,Ltd.Concurrency Computat.:Pract.Exper.2005;17:117–141PERFORMANCE TECHNOLOGY FOR COMPONENT SOFTWARE121Second,component communication mechanisms should not be dictated but be selectable based on what best suits encapsulated functionality and component–component proximity(e.g.same memory space,same cluster,local network or wide-area).Third,parallelism between components should not be restricted.This regards primarily support for parallel communication between components, but also relates to parallel component computing paradigms,as in the Single Component Multiple Data(SCMD)model[1],MxN coupling[14],and the macro-level dataflow Uintah Computational Framework(UCF)[15].Lastly,it should be possible to select component instances and configure component compositions for performance purposes,both prior to and during execution.2.2.Performance engineering and component softwareThe above important performance-related aspects of component-based scientific computing motivate fundamental performance engineering questions that are examined specifically in our research.•How are the performance of components and component compositions evaluated?•How can this evaluation be done robustly given the diversity of component(component composition)types and implementations?•How are component performance data represented and made available to the component framework as a whole?•How can the performance of component compositions be modeled?•What restrictions does a component approach place on performance engineering,and can the component architecture,framework infrastructure,and technologies be leveraged to implement and deliver performance engineering support more effectively?•Will the integration of performance engineering technology lead to a quantifiable improvement in performance of component-based scientific computing and how will this be demonstrated?•Is it possible to develop a component performance engineering solution that is general purpose and can be applied in many scientific computing contexts?We address these questions from the perspective of the technology to deliver performance-engineered solutions for scientific computing environments.2.3.Related workSince commercial component models are targeted mostly at serial computing environments, the design of performance methods and metrics for these software systems are not likely to account for critical requirements important to high-performance scientific computing such as memory hierarchy performance,data locality,orfloating point operation.Likewise,the distributed frameworks/component models(e.g.DCOM and CORBA)use commodity networking to connect components together,and lack consideration for high-performance network communication required for HPC.In a distributed environment,metrics such as round-trip time and network latency are often considered useful,while quantities such as bisection bandwidth,message-passing latencies and synchronization cost,which form the basis of much of the research in scientific performance evaluation, are left unaddressed.Copyright c 2005John Wiley&Sons,Ltd.Concurrency Computat.:Pract.Exper.2005;17:117–141122 A.MALONY ET AL.However,despite the different semantics,several research efforts in these standards offer viable strategies in measuring performance.A performance monitoring system for the Enterprise Java Beans standard is described in[16].For each component to be monitored,a proxy is created using the same interface as the component.The proxy intercepts all method invocations and notifies a monitor component before forwarding the invocation to the component.The monitor handles the notifications and selects the data to present,either to a user or to another component(e.g.a visualizer component). The goal of this monitoring system is to identify hot spots or components that do not scale well.The Wabash tool[17,18]is designed for pre-deployment testing and monitoring of distributed CORBA systems.Because of the distributed nature,Wabash groups components into regions based on the geographical location.An interceptor is created in the same address space of each server object(i.e.a component that provides services)and manages all incoming and outgoing requests to the server.A manager component is responsible for querying the interceptor for data retrieval and event management.In the work done by the Parallel Software Group at the Imperial College of Science in London[19,20],the research is focused on Grid-based component computing.However,the performance is also measured through the use of proxies.Their performance system is designed to automatically select the optimal implementation of the application based on performance models and available resources.With n components,each having C i implementations,there is a total ofn i=1C i implementations to choose from.The performance characteristics and a performance modelfor each component is constructed by the component developer and stored in the component repository. Their approach is to use the proxies to simulate an application in order to determine the call-path. This simulation skips the implementation of the components by using the proxies.Once the call-path is determined,a recursive composite performance model is created by examining the behavior of each method call in the call-path.In order to ensure that the composite model is implementation-independent,a variable is used in the model whenever there is a reference to an implementation. To evaluate the model,a specific implementation’s performance model replaces the variables and the composite model returns an estimated execution time or estimated cost(based on some hardware resources model).The implementation with the lowest execution time or lowest cost is then selected and a execution plan is created for the application.3.PERFORMANCE ENGINEERING METHODOLOGYBuilding efficient scientific applications as hierarchical compositions of cooperating components depends significantly on having a thorough knowledge of component performance and component interactions.To make effective decisions concerning component configuration,deployment,and coupling,it is important to develop a performance engineering methodology that complements the component and application development and execution processes.Ideally,the methodology would be supported by performance technology(measurement,analysis,and modeling)that extends the programming and execution environment to be performance observable and performance aware. However,the diversity of component functionality and the complexity of component implementation (e.g.different languages,hardware platforms,and parallelism modes)challenge performance technology to offer(more)robust solutions.Our objective here is to define a consistent performance engineering model for components and component ensembles,and to implement that model in an integrated fashion throughout the component development and execution framework.Copyright c 2005John Wiley&Sons,Ltd.Concurrency Computat.:Pract.Exper.2005;17:117–141PERFORMANCE TECHNOLOGY FOR COMPONENT SOFTWARE123Figure2.Performance engineered component design.ponent performance engineeringWhile component architectures facilitate the development of complex applications by allowing the independent creation of generic,reusable components and by easing their composition,application-level performance engineering requires knowledge about a component’s performance in different contexts and observation of the component’s performance behavior during execution.We view a performance-engineered component as having four constituent parts:•performance characterization and modeling(performance knowledge);•integrated performance measurement and analysis(performance observation);•runtime access to performance information(performance query);•mechanisms to alter component performance behavior(performance control).In Figure2,we show how a component’s design may be extended to support these performance engineering features.The intent is to keep the extended(performance engineered)design relatively consistent with the CCA model.We represent a component’s generic architecture by the middle(solid)box in thefigure. As shown,we distinguish between a component’s‘core’functionality and its variant sub-parts(both functional variants and code variants)that may be selected prior to or during component execution‡.‡This view is consistent with how components are regarded in several systems.(Note,a component’s ports can also have variants, but this is not drawn to reduce diagram complexity.)Copyright c 2005John Wiley&Sons,Ltd.Concurrency Computat.:Pract.Exper.2005;17:117–141124 A.MALONY ET AL.A component’s generic design can be‘performance engineered’with support for performance knowledge and/or performance observation.Performance knowledge extensions(left(dashed)box) provide means to describe and store what is‘known’about the component’s performance.This can take the form of empirical characterizations in a performance database,as well as performance models captured from empirical studies or analytical performance analysis.Performance observation extensions(right(dashed)box)implement performance measurement and analysis capabilities that are used during component execution.In addition,support for querying the performance information and for effecting control based on performance feedback complement the performance knowledge and observation parts.To justify these extensions,let usfirst consider the use of performance knowledge in the performance engineering of component frameworks.The ability to save information about a component’s performance allows that knowledge to be used for performance-guided component selection,deployment,and runtime adaptation.However,the representation of performance knowledge must be in common forms and there must be standard means to access selective performance information.In Figure2,we identify a‘component performance repository’where the performance knowledge could be kept and queried within the component framework,similar in concept(and, likely,function)to the CCA component repository.In fact,we could view the performance knowledge extension as a component in its own right,as suggested in thefigure by the separation of the left (dashed)box from original component.In this way,the performance knowledge component(PKC) could provide PKC ports that give component-level access to the performance information both to other components within the framework(black arrowhead)as well as back to the original component (shaded arrowhead)whose performance it represents.These‘feedback’port connections would allow for both static(instantiation-time)and dynamic(runtime)component control.The PKCs could also be instantiated and used,in particular,as active components to build runtime composite performance models for resource allocation and scheduling.Similarly,let us consider justification for performance observation support in our performance engineered component model.The ability to observe a component’s execution time performance is important for two reasons.By definition,any empirically derived performance knowledge requires some form of performance measurement and analysis.However,it does not mean that this support necessarily must be integrated in the component’s design.The second reason is to monitor the performance of a component during execution and use that information in dynamic performance decision making.Here,integrated performance observation support(measurement and analysis) is key.As depicted in Figure2(block-filled double arrow),this integration requires component instrumentation(both core and variant),runtime measurement and data collection,and online and/or offline performance analysis.In this respect,we can view performance observation in the performance engineered component model as a functional extension of the original component design.To allow the component framework to query performance behavior,this extension could include new component methods and ports(black arrowhead).It is also useful for the original component to be able to access its own performance observations to make internal performance decisions.An interesting design issue here concerns how this functionality should be supported.Our view is to further generalize the model to regard observation support as encapsulated in a performance observation component(POC)that is tightly coupled and co-resident with the original component.Special POC‘provides’ports allow the original component to use optimized interfaces(shaded arrowhead)to access‘internal’performance observations.Copyright c 2005John Wiley&Sons,Ltd.Concurrency Computat.:Pract.Exper.2005;17:117–141PERFORMANCE TECHNOLOGY FOR COMPONENT SOFTWARE 125}hierarchical component composition performance awareness monitoring}Figure 3.Hierarchical performance awareness monitoring for component composition.ponent composition performance engineering Our discussion above focused on the performance engineering model at the component level.Now we turn attention to component composition.The performance of component-based scientific applications depends as much (perhaps more)on the interplay of component functions and the computational resources used for component execution and interoperation,as it does on individual component performance.Because component operations and interactions can be complex and dynamic,and resource availability may be subject to variation,the management of component compositions throughout their execution becomes a critical element of their successful deployment and use.In general,this management can include resource assignment,component placement and scheduling,data communications control,and file storage and allocation.However,the issue for us is not necessarily to design the specific tools that will be part of the performance management solution,but to identify key technological capabilities needed to support the performance engineering of component ensembles.In practice,this distinction is blurred,admittedly due to the different infrastructure used for scientific component computing,ranging from wide-area Grids of heterogeneous resources to tightly-coupled large-scale computing systems.Thus,we restrict our attention to only two model concepts:performance awareness and performance attention .Performance awareness of component compositions relates to an ensemble view of performance,how this information is obtained,and how it can be accessed.As with components,performance engineering looks at composition performance knowledge and position performance knowledge can come from empirical as well as analytical evaluation,can utilize information provided at the component level,and can be stored in repositories for future review.Performance awareness extends the notion of component observation to ensemble-level performance monitoring .The idea is to associate monitoring components with levels of hierarchical component grouping,building upon component-level observation support.This is presented in Figure 3.These monitoring componentsCopyright c 2005John Wiley &Sons,Ltd.Concurrency Computat.:Pract.Exper.2005;17:117–141。