About The Project Summer Intern (July-Aug)

  • 格式:pdf
  • 大小:809.02 KB
  • 文档页数:22

Summer Intern (July-Aug) Report by Deepak Kumar Agarwal

About The Project: In .NET-DapPerAppCG, comprehensive set of solutions will be developed to efficiently manage and utilize input and intermediate parallel data for adaptive parallel web services executing on computational Grid resources. Various .NET mechanisms will be utilized for achieving the goals of the project, namely, UDDI for managing and discovery of parallel data distributed across Grid resources, Microsoft SQL server for maintaining metadata about the scattered data segments and SOAP-based protocols for remote data staging. Since in the current scientific applications, large-size data are considered to be the major bottlenecks in the applications' performance, the solutions that will be developed in the project will significantly reduce and/or eliminate the data movement costs thereby improving the overall performance of the parallel web services.

Objective: Grid Computing is today employed for various scientific applications. It is often the requirement of various organizations to compute large scale Scientific Parallel problem which gets restricted due to limited computational resources. But, consider a situation where the parallel problem is remotely invoked, the input data for the problem is staged from the web client's host to the remote computational resources. As the input data can be quite large its data staging needs to be handled efficiently.

We plan to use Web-Services for data staging. To avoid re-staging of data on repeat invocation of parallel application with the same data, the metadata for the input data and some associated data-structures will be stored in a SQL Database. Thus on re-invocation the SQL Server will be queried to find the associated data-structures and thus save re-transmission of data over the network.

The major challenge in this Web Service application is the size of the input data. The size of the input data can really be huge. With the worldwide acceptance of XML as the data transfer standard, we explore the possibility of using SOAP-based Web-Services for remote data staging. We have compared different implementations of web-service model for data staging and also the latest WSE tool support for data staging.

Birds’ eye view of client: Web Services - Behind the screen: We need to use three web-services for the whole process: i) string upload ( string filename, int count) This web service will enable the client to upload his job_file into data store residing at the server. The count parameter will act like a counter and help in synchronizing the staged data at the server and the actual data submitted at the client. The return value of this service is just a status value of success or failure. ii) string spawnmpi ( string machinename ) This web service does the real processing after the data is uploaded into the server’s workplace. The parameter machinename carries in the names of the parallel machines that will be used for processing the job. The client need not wait for the job to finish, the web service will return him a unique job_id . iii) string probe ( int job_id) This is the status query web service that allows client to look into the results of his job processing. The client has to submit the identification job_id to service and it will either return with an intelligent result of the corresponding job processing or with a request to wait for the job in queue. Choice for .Net Framework at the Client side: It was not difficult to arrive at the decision to adopt ASP.net as the Client interface. .NET framework provides inbuilt functionality for almost all sorts of operations form web-interfacing, database connectivity, flexible programming language support to implementation of cryptographic hash algorithms used here for data staging. It also provides tool like WSE for securing and scaling web services hosted by ASP.net client.

Choice for gSOAP at the Sever side: It was difficult to choose an approach for building up a server at the big-endian Solaris side as it involved lot of processing, logic, interoperability and scalability. But, gSoap2.7 toolkit came out to be a perfect choice. The gSOAP compiler tools provide a SOAP/XML-to-C/C++ language binding to ease the development of SOAP/XML Web services and client application in C and/or C++. gSOAP provides a C/C++ transparent SOAP API through the use of compiler technology that hides irrelevant SOAP-specific details from the user. The gSOAP stub and skeleton compiler automatically maps native and user-defined C and C++ data types to semantically equivalent XML data types and vice-versa. As a result, full SOAP interoperability is achieved with a simple API. The compiler enables the integration of (legacy) C/C++ and Fortran codes (through a Fortran to C interface), embedded systems, and real-time software in SOAP applications that share computational resources and information with other SOAP applications, possibly across different platforms, language environments, and disparate organizations located behind firewalls. gSOAP minimizes application adaptation for building Web Services. The gSOAP compiler generates SOAP marshalling routines that (de)serialize application-specific C/C++ data structures. gSOAP includes a WSDL generator to generate Web service descriptions for our Web services. The gSOAP WSDL parser and importer ”closes the circle” in that it enables client development without the need for users to analyze Web service details to implement a client or a server.