What Where How Many Combining Object Detectors and CRFs

  • 格式:pdf
  • 大小:810.92 KB
  • 文档页数:14

What,Where&HowMany?CombiningObjectDetectorsandCRFs

L’uborLadický,PaulSturgess,KarteekAlahari,ChrisRussell,andPhilipH.S.Torr⋆OxfordBrookesUniversityhttp://cms.brookes.ac.uk/research/visiongroup

Abstract.Computervisionalgorithmsforindividualtaskssuchasobjectrecog-nition,detectionandsegmentationhaveshownimpressiveresultsintherecentpast.Thenextchallengeistointegrateallthesealgorithmsandaddresstheprob-lemofsceneunderstanding.Thispaperisasteptowardsthisgoal.Wepresentaprobabilisticframeworkforreasoningaboutregions,objects,andtheirattributessuchasobjectclass,location,andspatialextent.OurmodelisaConditionalRan-domFielddefinedonpixels,segmentsandobjects.Wedefineaglobalenergyfunctionforthemodel,whichcombinesresultsfromslidingwindowdetectors,andlow-levelpixel-basedunaryandpairwiserelations.Oneofourprimarycon-tributionsistoshowthatthisenergyfunctioncanbesolvedefficiently.Exper-imentalresultsshowthatourmodelachievessignificantimprovementoverthebaselinemethodsonCamVidandPASCALVOCdatasets.

1IntroductionSceneunderstandinghasbeenoneofthecentralgoalsincomputervisionformanydecades[1].Itinvolvesvariousindividualtasks,suchasobjectrecognition,imageseg-mentation,objectdetection,and3Dscenerecovery.Substantialprogresshasbeenmadeineachofthesetasksinthepastfewyears[2–6].Inlightofthesesuccesses,thechal-lengingproblemnowistoputtheseindividualelementstogethertoachievethegrandgoal—sceneunderstanding,aproblemwhichhasreceivedincreasingattentionre-cently[6,7].Theproblemofsceneunderstandinginvolvesexplainingthewholeim-agebyrecognizingalltheobjectsofinterestwithinanimageandtheirspatialextentorshape.Thispaperisasteptowardsthisgoal.Weaddresstheproblemsofwhat,where,andhowmany:werecognizeobjects,findtheirlocationandspatialextent,seg-mentthem,andalsoprovidethenumberofinstancesofobjects.Thisworkcanbeviewedasanintegrationofobjectclasssegmentationmethods[3],whichfailtodis-tinguishbetweenadjacentinstancesofobjectsofthesameclass,andobjectdetectionapproaches[4],whichdonotprovideinformationaboutbackgroundclasses,suchasgrass,skyandroad.Theproblemofsceneunderstandingisparticularlychallenginginscenescomposedofalargevarietyofclasses,suchasroadscenes[8]andimagesinthePASCALVOC2L.Ladicky,P.Sturgess,K.Alahari,C.Russell,andP.H.S.Torr(a)(b)(c)(d)Fig.1.Aconceptualviewofourmethod.(a)Anexampleinputimage.(b)Objectclasssegmen-tationresultofatypicalCRFapproach.(c)Objectdetectionresultwithforeground/backgroundestimatewithineachboundingbox.(d)Resultofourproposedmethod,whichjointlyinfersaboutobjectsandpixels.StandardCRFmethodsappliedtocomplexscenesasin(a)underperformonthe“things”classes,e.g.inaccuratesegmentationofthebicyclistandpersons,andmissesapoleandasign,asseenin(b).However,objectdetectorstendtoperformwellonsuchclasses.Byincorporatingthesedetectionhypotheses(§2.2),shownin(c),intoourframework,weaimtoachieveanaccurateoverallsegmentationresultasin(d)(§3.3).(Bestviewedincolour)

dataset[9].Forinstance,roadscenedatasetscontainclasseswithspecificshapessuchasperson,car,bicycle,aswellasbackgroundclassessuchasroad,sky,grass,whichlackadistinctiveshape(Figure1).Thedistinctionbetweenthesetwosetsofclasses—referredtoasthingsandstuffrespectively—iswellknown[10–12].Adelson[10]emphasizedtheimportanceofstudyingthepropertiesofstuffinearlyvisiontasks.Recently,theseideasarebeingrevisitedinthecontextofthenewvisionchallenges,andhavebeenimplementedinmanyforms[12–15].Inourwork,wefollowthedefinitionbyForsythetal.[11],wherestuffisahomogeneousorreoccurringpatternoffine-scaleproperties,buthasnospecificspatialextentorshape,andathinghasadistinctsizeandshape.Thedistinctionbetweentheseclassescanalsobeinterpretedintermsoflocalization.Things,suchascars,pedestrians,bicycles,canbeeasilylocalizedbyboundingboxesunlikestuff,suchasroad,sky1.Completesceneunderstandingrequiresnotonlythepixel-wisesegmentationofanimage,butalsoanidentificationofobjectinstancesofaparticularclass.Consideranimageofaroadscenetakenfromonesideofthestreet.Ittypicallycontainsmanycarsparkedinarow.Objectclasssegmentationmethodssuchas[3,8,16]wouldlabelallthecarsadjacenttoeachotherasbelongingtoalargecarsegmentorblob,asillustratedinFigure2.Thus,wewouldnothaveinformationaboutthenumberofinstancesofaparticularobject—carinthiscase.Ontheotherhand,objectdetectionmethodscanidentifythenumberofobjects[4,17],butcannotbeusedforbackground(stuff)classes.Inthispaper,weproposeamethodtojointlyestimatetheclasscategory,location,andsegmentationofobjects/regionsinavisualscene.Wedefineaglobalenergyfunc-tionfortheConditionalRandomField(CRF)model,whichcombinesresultsfromde-tectors(Figure1(c)),pairwiserelationshipsbetweenmid-levelcuessuchassuperpixels,andlow-levelpixel-basedunaryandpairwiserelations(Figure1(b)).Wealsoshowthat,unlike[6,18],ourformulationcanbesolvedefficientlyusinggraphcutbasedmove