FAULT-TOLERANT DATA SHARING FOR HIGH- LEVEL GRID PROGRAMMING A HIERARCHICAL STORAGE ARCHITE

  • 格式:pdf
  • 大小:252.15 KB
  • 文档页数:12

FAULT-TOLERANTDATASHARINGFORHIGH-LEVELGRIDPROGRAMMING:AHIERARCHICALSTORAGEARCHITECTURE

MarcoAldinucci,MarcoDaneluttoComputerScienceDepartment,UniversityofPisaLargoBrunoPontecorvo3,56127Pisa,Italy

aldinuc@di.unipi.it

GabrielAntoniu,MathieuJanINRIARennesCampusdeBeaulieu,35042Rennes,France

gabriel.antoniu@irisa.fr,mathieu.jan@irisa.fr

AbstractEnablinghigh-levelprogrammingmodelsongridsistodayamajorchallenge.Awaytoachievethisgoalreliesontheuseofenvironmentsabletotransparentlyandautomaticallyprovideadequatesupportforlow-level,grid-specificissues(fault-tolerance,scalability,etc.).Thispaperdiscussestheaboveapproachwhenappliedtogriddatamanagement.Asacasestudy,weproposea2-tiersoftwarearchitecturethatsupportstransparent,fault-tolerant,grid-leveldatasharingintheASSISTprogrammingenvironment(UniversityofPisa),basedontheJuxMemgriddatasharingservice(INRIARennes).

Keywords:Grid,sharedmemory,High-levelprogramming,memoryhierarchy,P2P.21.IntroductionGridcomputinghasemergedasanattempttoprovideuserswiththeillusionofaninfinitelypowerful,easy-to-usecomputer,whichcansolveverycomplexproblems.Thisveryappealingillusionistobeprovided(1)byrelyingontheaggregatedpowerofstandard(so,inexpensive),geographicallydistributedresourcesownedbymultipleorganizations;(2)byhidingasmuchaspossiblethecomplexityofthedistributedinfrastructuretousers.However,thecurrentstatusinmostsoftwaregridinfrastructuresavailabletodayisratherfarawayfromthisvision.Whendesigningprogramsabletorunonsuchlarge-scaleplatforms,programmersoftenneedtoexplicitlytakeintoaccountresourceheterogeneity,aswellastheunreliabilityofthedistributedinfrastructure.Inthiscontext,thegridcommunityconvergestowardsaconsensusabouttheneedforahigh-levelprogrammingmodel,whereasmostofthegrid-specificeffortsaremovedawayfromprogrammerstogridtoolsandrun-timesystems.Thisdirectioniscurrentlypursuedbyseveralresearchinitiativesandprogrammingenvironments,suchasASSIST[17],eSkel[9],GrADS[14],ProActive[8],Ibis[16],HigherOrderComponents[11],etc.,alongthelinesofCoreGRID’s“invisiblegrid”approachtonextgenerationgridprogrammingmodels.Inthiswork,weexploretheapplicabilityoftheaboveapproachtodatamanagementforhigh-levelgridprogramming.Weconsiderthreemainaspectsthatneedtobeautomaticallyhandledbythedatastorageinfrastructure:dataaccesstransparency,datapersistenceandstoragefault-tolerance.

Transparentaccesstodataacrossthegrid.Oneofthemajorgoalsof

thegridconceptistoprovideaneasyaccesstheunderlyingresources,inatransparentway.Theusershouldnotneedtobeawareofthelocalizationoftheresourcesallocatedtotheapplicationsubmitted.Whenappliedtotheman-agementofthedatausedandproducedbyapplications,thisprinciplemeansthatthegridinfrastructureshouldautomaticallyhandledatastorageanddatatransferamongclients,computingserversandstorageserversasneeded.How-ever,mostprojectscurrentlystillrelyontheexplicitdataaccessmodel,whereclientshavetomovedatatocomputingservers.Inthiscontext,grid-enabledfiletransfertoolshavebeenproposed,suchasGridFTP[5],DiskRouter[12],etc.Inordertoachievearealvirtualizationofthemanagementoflarge-scaledistributeddata,astepforwardhasbeenmadebyproposingatransparentdataaccessmodel,asapartoftheconceptofgriddata-sharingservice,illustratedbytheJuxMemsoftwareexperimentalplatform[6].

Persistentstorageofdataonthegrid.Sincegridapplicationstypically

handlelargemassesofdata,datatransferamongsitescanbecostly,intermsofbothlatencyandbandwidth.Therefore,thedata-sharingservicehastoprovidepersistentdatastorage.DataproducedbyonecomputationcanbeFault-TolerantDataSharingforHigh-levelGridProgramming3madeavailabletosomeothercomputationthroughdirectaccessviagloballyshareddataidentifiers,byavoidingrepeatedtransfersoflargevolumesofdatabetweenthedifferentcomponentsofthegrid.

Fault-tolerantstorageofgriddata.Datastorageonthegridmustcopewith

eventssuchasstorageresourcesjoiningandleaving,orunexpectedlyfailing.Replicationtechniquesandfailuredetectionmechanismsarethusnecessary,inordertoenhancedataavailabilitydespitedisconnectionsandfailures.Suchmechanismsneedtobecoupledwithcheckpoint/restarttechniquesforappli-cationcomponentswithdatareplicationmechanisms.Thisway,inreactiontofaults,checkpointedapplicationcomponentscouldbemigratedtoavailableresourcesandrestarted,aslongastheapplicationstatusandtheapplicationdataremainavailablethankstofault-tolerantstorage.Whiletheabovepropertiesaredesirableingeneralforgriddatasharing,inthispaperwerestrictourdiscussiontothemorespecificcaseofapplicationsbasedontheASSIST[3]programmingenvironment,developedattheUniver-sityofPisa.ASSISTprovidesaprogrammingmodelthatenablestransparentdatasharingamongdistributedexecutionentities.However,itsdedicatedstor-agecomponent(calledad-HOC[4])doesnotenforcedatapersistenceandisnottoleranttonodefailures.SuchfeaturesareexhibitedbytheJuxMem[6]griddata-sharingservicedevelopedatINRIARennes,whichenablestransparentaccesstogrid-scalestorage,basedonP2Ptechniques.ThegoalofthispaperistostudyhowASSISTandJuxMemcouldbeintegrated,inordertobettersupportahigh-levelgridprogrammingapproach.