SPL_2012_SK-SVD
- 格式:pdf
- 大小:111.51 KB
- 文档页数:10
1StagewiseK-SVDtodesignefficient
dictionariesforsparserepresentationsCristianRusuandBogdanDumitrescu,Member,IEEE
AbstractTheproblemoftrainingadictionaryforsparserepresentationsfromagivendatasetisreceivingalotofattentionmainlyduetoitsapplicationsinthefieldsofcoding,classificationandpatternrecognition.Oneoftheopenquestionsishowtochoosethenumberofatomsinthedictionary:ifthedictionaryistoosmallthentherepresentationerrorsarebigandifthedictionaryistoobigthenusingitbecomescomputationallyexpensive.Inthispaperwesolvetheproblemofcomputingefficientdictionariesofreducedsizebyanewdesignmethod,calledStagewiseK-SVD,whichisanadaptationofthepopularK-SVDalgorithm.SinceK-SVDperformsverywellinpractice,weuseK-SVDstepstograduallybuilddictionariesthatfulfillanimposederrorconstraint.Theconceptualsimplicityofthemethodmakesiteasytoapply,whilethenumericalexperimentshighlightitsefficiencyfordifferentovercompletedictionaries.
IndexTermsSparserepresentations,dictionarylearning,K-SVDEDICS:DSP-TFSR,IMD-MDSP
Copyright(c)2012IEEE.Personaluseofthismaterialispermitted.However,permissiontousethismaterialforanyotherpurposesmustbeobtainedfromtheIEEEbysendingarequesttopubs-permissions@ieee.org.TheauthorsarewiththeDepartmentofAutomaticControlandComputers,UniversityPolitehnicaofBucharest,Spl.Independent¸ei313,Bucharest060042,Romania(e-mails:cristian.rusu@schur.pub.ro,bogdan.dumitrescu@acse.pub.ro).B.DumitrescuisalsowithDepartmentofSignalProcessing,TampereUniversityofTechnology,33101Tampere,Finland(e-mail:bogdan.dumitrescu@tut.fi).
July18,2012DRAFT2I.INTRODUCTION
Thefieldofsparserepresentationshasreceivedcontinuousattentionduringthelatestyears,fueledbyrecenttheoreticaldevelopments[1][2]andapplications[3][4].Inthiscontext,theproblemofdesigningbasesforsparserepresentationsiscentralandhasamultitudeofapplicationareas,including:image[5]andvideo[6]coding,patternrecognitionandclassification[7].Thegoalistofindovercompletebases(dictionaries)inwhichsignalshavemultiplerepresentationsandweseekthesparsestlinearcombinationthatgivesagoodapproximationofthesignal.Themainadvantagethatthisdesignphilosophybringsisthepossibilitytocreateextremelygoodsparserepresentationsfortheinputdata,ofcourse,atthecostofahighercomputationaltime.GivenadatasetY∈Rn×mtheproblemcanbeformulatedinthefollowingmanner:minimizeA,X||Y−AX||
2
F
subjectto||xi||0≤s0,1≤i≤m
(1)
thatreads:findthefullmatrixA∈Rn×NwithN≪m,calledheredictionaryanditscolumnscalledatoms,andthesparserepresentationmatrixX∈RN×mwithtargetsparsitys0suchthatY≈AX,where||E||2F=ni=1m1E2ijistheFrobeniusnormand||xi||0isthenumberofnonzeroelementsofthei-thcolumnofX.Fortherepresentationperformanceofthealgorithmsdescribedinthispaperweconsidertherootmeansquarederror(RMSE),ameasurethatisbuiltupontheFrobeniusnorm.ApopularandveryefficientapproachtosolvethisproblemistheK-SVDalgorithm[8].Thisalgorithmemploysanalternateoptimizationprocessintwosteps:keepingfixedthedictionaryA,createthesparserepresentationmatrixXusingtheorthogonalmatchingpursuit(OMP)algorithm[9]andtheninthesecondstep,updateboththedictionaryandtherepresentationsusingarank-1approximationobtainedbythesingularvaluesdecomposition(SVD).Inordertospeeduptheprocedure,OMPisimplementedina”bulk”fashionandtheSVDstepcanbeapproximatedwithafewiterationsofthepowermethod[10].Eventhoughtheresultsdependstronglyontheinitialization,ingeneralthisapproachprovidesverygoodresultsinareasonableamountoftime.Intermsoftherepresentationperformance,itisclearthatabiggerdictionaryisbetter,meaningthatitshouldreachalowerrepresentationerrorthananystrictlysmallerdictionary.Sinceproblem(1)isnotconvex,thisdoesnotalwayshappeninpracticaldesigns.ThereisnogeneralprocedureandtherearenoguidelineswhenconsideringthevaluesofN.MakingtheobviouschoiceofpickingaverylargeNisnotgoodsincelargedictionaries,eveniftheyprovideaverygoodpresentationerror,arejustnotfeasibleinmanyrealworldapplications.Ingeneral,whenK-SVDisappliedinpractice,thesizeNis
July18,2012DRAFT3chosenbyaheuristicorafterconductingseveralnumericalexperimentsbecausethereisanimportantrepresentationerror/speedtradeoffthattakesplace.Consequently,thetaskoffindingagoodvalueofNwhentherestrictionsontherepresentationerrorsareknownisofgreatpracticalimportanceandtheK-SVDmethoddoesnotaddressitinanyway.Previousworkrelatedtotheconstructionofsize-optimizeddictionariesincludes:aK-SVDapproachthatgraduallyprunestheunder-utilizedorhighlycorrelatedatomsbyusingacompetitiveagglomerationalgorithmapproach[11]oronethatalsoincorporatesasubtractiveclusteringmethodtokeepthemostimportantatomswhileitalsoprunestheredundantones[12],theadditionofcriteriaforeliminatingredundantatomsbydiscardinglinearlydependentatomsandaddinglinearindependentfeatures[13],selectingtheatomsofthedictionaryfromafixedsetofcandidatedictionariesbyagreedymethod[14]andsegmentationofthetrainingdataintosubspacesandconstructingthedictionarybyextractingsimilarfeaturesfrommultiplesubspaces[15].InthispaperweproposeanewwayofconstructingthedictionariesthatreliesonK-SVDstepsbutalsoaddressestheissueofthesizeN.Thedictionaryisbuiltfromthegroundup,startingwithonlys