Research Statement

Research Statement Debzani Deb

My research addresses the problems associated with the performance, scalability and management of distributed systems. I am particularly interested in developing techniques that enable self-managed, scalable and efficient deployment of large-scale scientific and engineering applications in a highly dynamic and unpredictable distributed environment. Next few sections describe some of my major research accomplishments to date and plans for future work.

Summary of Current Research My graduate research is motivated by the growing popularity and performance needs of large-scale scientific and engineering applications. Many scientific fields, such as genomics, astrophysics, geophysics, computational neuroscience or bioinformatics require massive computational power and resources and can benefit from a large-scale integrated infrastructure, formed by harnessing the spare compute cycles of distributed computation and communication resources. Typically these applications composed of a large number of distributed components and it is important to meet the computational power and network bandwidth requirements of those components and their interactions. However satisfying these requirements in a large-scale, heterogeneous, and highly dynamic distributed environment is a significant challenge. As systems and application grow in scale and complexity, attaining the desired level of performance in this uncertain environment using current approaches based on global knowledge, centralized scheduling and manual reallocation becomes infeasible. Application components therefore need to be self-managed in order to be able to dynamically detect and respond, quickly and correctly, to changes in the application’s behaviors and the state of the underlying environment.

I have designed and implemented algorithms and mechanisms for achieving self-managed deployment of computationally intensive scientific and engineering applications in highly dynamic and unpredictable distributed environment. My approach is to incorporate autonomic entities [1] to handle the complexities associated with distribution, configuration, coordination and efficient execution of application components and to adapt to the changes in application behavior and the underlying environment. Through analysis and experiments, I have shown that such an automated approach results in higher utilization of distributed resources while meeting application performance demands. As part of my thesis, I have addressed several important problems that arise in the design of self-managed deployment. In particular, I have developed techniques for automatically identifying application components and their estimated resource requirements within a centralized application and used them in order to model the application into a graph abstraction. I also developed techniques that allow the distributed resources to self-organize in a utility-aware way while assuming minimal knowledge about the system. Finally, to achieve efficient initial deployment of application components to the network nodes and their run-time reconfigurations, I have designed a scalable and adaptive scheduling algorithm. The scheduling is governed by a specific utility function that returns the overall system’s utility based on different application and system level attributes. The main contributions of my Ph.D. research are detailed in the following few sections.

Application Components and Their Resource Requirements [2] Analyzing and representing application in terms of its components and their internal dependencies is important in order to provide the self-managing capabilities because this is actually the system’s view of the run-time structure of an application. In order to satisfy application performance goals, it is also important to infer the resource requirement of application components and their links so that an efficient mapping of components to resources can be achieved. Both of these objectives are difficult to accomplish. To deduce the application components and their interdependencies, I have used a modified and extended version of Spiegel’s algorithm [3] that statically analyzes the centralized Java source code and produces a directed graph where each node and edge represents objects and relations (create, read-only, write and reference) among them respectively. This approach has the advantage of detecting object-level interactions, rather than detecting class-level dependencies performed by most other automatic application partitioning systems. I have performed the static analysis on top of the Jimple [4] representation, which is part of the Soot [4] framework, a set of Java APIs for manipulating and optimizing Java byte code. My approach is to read all class files that are required by the application starting with the main method and as each class is read, convert it into Jimple representation, suitable for analysis and transformations. To address the problem of inferring computational and communication resource requirements of the application components and their links, I have adopted several techniques based on measurements and heuristics that estimate the resource requirements by analyzing the Jimple representations of the class files. The resource requirements are estimated at the same time while the application graph is being constructed, and therefore this approach does not incur additional cost due to repetitive analysis of the application source code. Once the application components and their resource requirements are identified, I have modeled each application as a node-weighted, edge-weighted directed graph, where nodes and edges represents the objects and their interactions respectively and the weights associated with nodes and edges signify the amount of computation and communication needed for the corresponding objects and their links.