Information
- 格式:pdf
- 大小:222.65 KB
- 文档页数:11
Informationretrievalonmixedwrittenandspokendocuments
BenoitFavre,PatriceBellot,Jean-Fran¸coisBonastre{benoit.favre,patrice.bellot,jean-francois.bonastre}@lia.univ-avignon.frLaboratoired’Informatiqued’Avignon-Universit´ed’Avignon339,chemindesMeinajaries-AgroparcBP122884911AVIGNONCedex9-FRANCET´el:+33(0)490843509Fax:+33(0)490843501
January5,2004
AbstractWhileadvanceshavebeenmadeinstructuring,indexingandretrievalofmultimediadocuments,weproposetostudytheunexploredproblematicsofinformationretrievalonheterogeneousmediasetscomposedofwrittenandspokendocuments.Thecoverageofmodalitiesinretrievedresultsseemstobeanimportantpartoftheuser’sinformationneed.Weshowthatthisproblematicisnotsatisfiedbytheusualbag-of-wordsmodelsandproposeamethodtobalancemodalitieswithinthequeryexpansionprocessoftheprobabilisticmodel.Astherehasneverbeenexperimentsinthisdomain,wesuggestthatbuildingevaluationdatafortheaddressedmedias(textandspeech)aswellasothermedias(image...)isimportantforthemultimediainformationretrievalcommunity.
1IntroductionTheamountofinformationavailableovernetworksgrowseverydays.Thisinformationworthsbee-ingaccessedandstructured.Indexationandinformationretrievalareessentialtaskstorealizetheseobjectives.
Majoradvancesintextualinformationretrievalwhereobservedwithinthelastyears.Anewneedformultimediainformationretrievalbringingnewproblematicsisimpliedbytherisingproductionofmultimediadocuments,thegrowthofcapacities,ratesandcomputationpower.
Multimediainformationiscomposedofimages,videosandaudiostreamsinadditiontotextualdocuments.Whereashighlevelinformationextractionfromstillandanimatedimagesareoutbreaking,automaticspeechrecognitionisusedtotranscribeandindexthespokencontentofmultimediadocumentsandperformswellenoughtoachieveSpokenDocumentRetrieval(SDR).SpeechindexinghasbeenmuchstudiedduringtheSDRNIST(NationalInstituteofScienceandTechnology)evaluationcampains[GAV00].
Weproposetostudythebehaviourofclassicalinformationretrievalmethodsonmultimediadocumentcollectionscomposedoftextualdocumentsandspeechtranscripts.Thismultimediainformationretrievaltaskdealswithheterogeneousdocumentsetsand,asfarasweknow,hasneverbeenexploredbefore.Wenoticethattheuser’sinformationneedimpliestheconsiderationofthemodalitycoverageinthesearchresults.Thisleadstoabalanceproblembetweenmodalitiesthatcanberesolvedusinganad-hocqueryexpansionmethodthatweprovideinsection5.2.1.Weconcludeontheneedforevaluationdataforthisnewkindofinformationretrieval.
12MotivationsDuringthelastyears,informationretrievalhasbeenstudiedonseparatedmedia,butasmultimediadocumentssurroundus,themediasarenowstudiedtogether.Linksaremadeintheinformationspacebetweenmedia.Forinstance,imageretrievalusinglowlevelfeatureslikecolorhistogramsandtexturegiveslowretrievalefficiency[SC01].Therefore,textiscapturedaroundanimage(caption...)togethighlevelconceptsrelatedtotheimage.Thiskindofinformationretrievalbindsmultiplemediasinsingledocumentsand,togetbacktoourpreviousexample,animagewon’tberetrievedwithoutitscaption.
Westudyinthisarticleanapproachofmultimediainformationretrievalwherethedocumentcollec-tionismadeofdocumentsofmultiplemedias.Thestudyisreducedtotextualdocumentsandspeechtranscriptsbecausethesemodalitieshavebeenwellstudiedduringthepastyearsandmayberetrievedusingsimilarmethods.Textualdocumentsandspeechtranscriptssharethesamerepresentationofthecontent(words,sentences...)butthelaterenablestheretrivalofaudiodocuments.Weareparticularlyinterestedinknowinghowtheuserwoulddealwithamultimediadatabasecontainingtextualandaudioinformation.Theusermaybelookingfortextualonlyoraudioonlydocumentsandonthecontraryhemaywanttocomparetextualtoaudiocontent.Henceaninformationretrievalsystemworkingonthiskindofdatabasewillhavetodealwithtwowaysofformulatingtheuserinformationneed.
Informationretrievalonmixedmediacorpusisanimportantsteptowardmulitmediainformationretrievalanddoesnotseem(asfarasweknow)tohavebeenstudiedbefore.
3Relatedwork3.1TextualinformationretrievalThereareseveralapproachestoinformationretrievalgiventhestudiedmedia.Textretrievalhasbeenthefirstapprochtoinformationretrievalandemergedfromadifficultproblematic:findthedocumentsmeetingtheuser’sinformationneed.Thisneedisnotwelldefinedandbiasedlyexpressedviaatextualquery.Adocumentwillberelevant(ornotrelevant)totheuser’squerywhenheliked(disliked)it.Thiswayofpartitionningsearchresultshelpsinevaluatinganinformationretrievalsystemusingprecisionandrecallmetrics.Theprecisionisthenumberofrelevantdocumentsretrievedcomparedtothenumberofretrieveddocumentsandtherecallisthenumberofrelevantdocumentsretrievedcomparedtothenumberofrelevantdocumentsinthetargetcollection.