SPL_2012_SK-SVD

  • 格式:pdf
  • 大小:111.51 KB
  • 文档页数:10

1StagewiseK-SVDtodesignefficient

dictionariesforsparserepresentationsCristianRusuandBogdanDumitrescu,Member,IEEE

AbstractTheproblemoftrainingadictionaryforsparserepresentationsfromagivendatasetisreceivingalotofattentionmainlyduetoitsapplicationsinthefieldsofcoding,classificationandpatternrecognition.Oneoftheopenquestionsishowtochoosethenumberofatomsinthedictionary:ifthedictionaryistoosmallthentherepresentationerrorsarebigandifthedictionaryistoobigthenusingitbecomescomputationallyexpensive.Inthispaperwesolvetheproblemofcomputingefficientdictionariesofreducedsizebyanewdesignmethod,calledStagewiseK-SVD,whichisanadaptationofthepopularK-SVDalgorithm.SinceK-SVDperformsverywellinpractice,weuseK-SVDstepstograduallybuilddictionariesthatfulfillanimposederrorconstraint.Theconceptualsimplicityofthemethodmakesiteasytoapply,whilethenumericalexperimentshighlightitsefficiencyfordifferentovercompletedictionaries.

IndexTermsSparserepresentations,dictionarylearning,K-SVDEDICS:DSP-TFSR,IMD-MDSP

Copyright(c)2012IEEE.Personaluseofthismaterialispermitted.However,permissiontousethismaterialforanyotherpurposesmustbeobtainedfromtheIEEEbysendingarequesttopubs-permissions@ieee.org.TheauthorsarewiththeDepartmentofAutomaticControlandComputers,UniversityPolitehnicaofBucharest,Spl.Independent¸ei313,Bucharest060042,Romania(e-mails:cristian.rusu@schur.pub.ro,bogdan.dumitrescu@acse.pub.ro).B.DumitrescuisalsowithDepartmentofSignalProcessing,TampereUniversityofTechnology,33101Tampere,Finland(e-mail:bogdan.dumitrescu@tut.fi).

July18,2012DRAFT2I.INTRODUCTION

Thefieldofsparserepresentationshasreceivedcontinuousattentionduringthelatestyears,fueledbyrecenttheoreticaldevelopments[1][2]andapplications[3][4].Inthiscontext,theproblemofdesigningbasesforsparserepresentationsiscentralandhasamultitudeofapplicationareas,including:image[5]andvideo[6]coding,patternrecognitionandclassification[7].Thegoalistofindovercompletebases(dictionaries)inwhichsignalshavemultiplerepresentationsandweseekthesparsestlinearcombinationthatgivesagoodapproximationofthesignal.Themainadvantagethatthisdesignphilosophybringsisthepossibilitytocreateextremelygoodsparserepresentationsfortheinputdata,ofcourse,atthecostofahighercomputationaltime.GivenadatasetY∈Rn×mtheproblemcanbeformulatedinthefollowingmanner:minimizeA,X||Y−AX||

2

F

subjectto||xi||0≤s0,1≤i≤m

(1)

thatreads:findthefullmatrixA∈Rn×NwithN≪m,calledheredictionaryanditscolumnscalledatoms,andthesparserepresentationmatrixX∈RN×mwithtargetsparsitys0suchthatY≈AX,where||E||2F=󰀁ni=1󰀁m1E2ijistheFrobeniusnormand||xi||0isthenumberofnonzeroelementsofthei-thcolumnofX.Fortherepresentationperformanceofthealgorithmsdescribedinthispaperweconsidertherootmeansquarederror(RMSE),ameasurethatisbuiltupontheFrobeniusnorm.ApopularandveryefficientapproachtosolvethisproblemistheK-SVDalgorithm[8].Thisalgorithmemploysanalternateoptimizationprocessintwosteps:keepingfixedthedictionaryA,createthesparserepresentationmatrixXusingtheorthogonalmatchingpursuit(OMP)algorithm[9]andtheninthesecondstep,updateboththedictionaryandtherepresentationsusingarank-1approximationobtainedbythesingularvaluesdecomposition(SVD).Inordertospeeduptheprocedure,OMPisimplementedina”bulk”fashionandtheSVDstepcanbeapproximatedwithafewiterationsofthepowermethod[10].Eventhoughtheresultsdependstronglyontheinitialization,ingeneralthisapproachprovidesverygoodresultsinareasonableamountoftime.Intermsoftherepresentationperformance,itisclearthatabiggerdictionaryisbetter,meaningthatitshouldreachalowerrepresentationerrorthananystrictlysmallerdictionary.Sinceproblem(1)isnotconvex,thisdoesnotalwayshappeninpracticaldesigns.ThereisnogeneralprocedureandtherearenoguidelineswhenconsideringthevaluesofN.MakingtheobviouschoiceofpickingaverylargeNisnotgoodsincelargedictionaries,eveniftheyprovideaverygoodpresentationerror,arejustnotfeasibleinmanyrealworldapplications.Ingeneral,whenK-SVDisappliedinpractice,thesizeNis

July18,2012DRAFT3chosenbyaheuristicorafterconductingseveralnumericalexperimentsbecausethereisanimportantrepresentationerror/speedtradeoffthattakesplace.Consequently,thetaskoffindingagoodvalueofNwhentherestrictionsontherepresentationerrorsareknownisofgreatpracticalimportanceandtheK-SVDmethoddoesnotaddressitinanyway.Previousworkrelatedtotheconstructionofsize-optimizeddictionariesincludes:aK-SVDapproachthatgraduallyprunestheunder-utilizedorhighlycorrelatedatomsbyusingacompetitiveagglomerationalgorithmapproach[11]oronethatalsoincorporatesasubtractiveclusteringmethodtokeepthemostimportantatomswhileitalsoprunestheredundantones[12],theadditionofcriteriaforeliminatingredundantatomsbydiscardinglinearlydependentatomsandaddinglinearindependentfeatures[13],selectingtheatomsofthedictionaryfromafixedsetofcandidatedictionariesbyagreedymethod[14]andsegmentationofthetrainingdataintosubspacesandconstructingthedictionarybyextractingsimilarfeaturesfrommultiplesubspaces[15].InthispaperweproposeanewwayofconstructingthedictionariesthatreliesonK-SVDstepsbutalsoaddressestheissueofthesizeN.Thedictionaryisbuiltfromthegroundup,startingwithonlys