Mapping perceived depth to regions of interest in stereoscopic images

MappingPerceivedDepthtoRegionsofInterestin

StereoscopicImages

NickHolliman

DepartmentofComputerScience,UniversityofDurham,Durham,UnitedKingdom

ABSTRACT

Theusableperceiveddepthrangeofastereoscopic3Ddisplayislimitedbyhumanfactorsconsiderationstoa

definedrangearoundthescreenplane.Thereisthereforeaneedinstereoscopicimagecreationtomapdepth

fromthescenetoatargetdisplaywithoutexceedingtheselimits.Recentimagecapturemethodsprovideprecise

controloverthisdepthmappingbutmapasinglerangeofscenedepthasawholeandareunabletogive

preferentialstereoscopicrepresentationtoaparticularregionofinterestinthescene.

Anewapproachtostereoscopicimagecreationisdescribedthatallowsadefinedregionofinterestinscene

depthtohaveanimprovedperceiveddepthrepresentationcomparedtootherregionsofthescene.Forexample

inagamethismaybetheregionofdeptharoundagamecharacter,orinascientificvisualizationtheregion

aroundaparticularfeatureofinterest.

Torealisethisapproachwepresentanovelalgorithmforstereoscopicimagecaptureanddescribeanimple-

mentationforthewidelyusedray-tracingpackagePOV-Ray.Resultsdemonstratehowthisapproachprovides

contentcreatorswithimprovedcontroloverperceiveddepthrepresentationinstereoscopicimages.

Keywords:GraphicsSystems,HumanFactors,Rendering,VirtualReality,Stereoscopic,3Ddisplay

1.INTRODUCTION

Thepropertiesofstereoscopicdisplaysfrequentlypresentachallengeforcontentcreatorswithmanyscenes

containingalargerdepthrangethana3Ddisplaycanpresenttotheviewer.Forexample,trialsusingahigh

qualitydesktopauto-stereoscopic3DdisplaybySharpCorporation1demonstratedthattherangeofavailable

perceiveddepthrangecouldbeaslittleas[−50mm,+60mm]in-frontandbehindthedisplaysurface.These

limitsaregenerallyunderstoodtoresultfromacombinationoffactorsincludingthedisplay’slevelofinter-

channelcrosstalk,2thequalityofthestereoscopicimages3andtheabilityoftheviewertofusescreendisparity

intoasingleimage.

Manymethodshavebeenproposedtocontrolhowscenedepthiscapturedinastereoscopicimage.The

problemcanbeconsideredamappingofscenedepthontotheavailableperceiveddepthrangeonthetarget

stereoscopicdisplayasshowninfigure1(a).Recentsolutionsareabletoguaranteethismappinghascertain

properties,forexamplethattheperceiveddepthwillnotexceedpre-definedrangesandthattheperceiveddepth

remainsconstantwhenthereisheadmovement.Thishasresultedinbenefitsforcontentcreatorswhocannow

definethescenedepthrangetheywanttocapturewhilebeingsurethatthiswillbereproducedasexpectedon

thetargetdisplay.Experiencegeneratingimagesusingoneofthesemethods1waspositive,itallowednovice

3Ddisplayuserstohaverepeatablecontroloverthedepthmappingfromscenetodisplayedimage.Asaresult

contentproductionusingcomputergraphicsanddigitalphotographynolongerrequiredrepeatedtrialanderror

toproduceanimagewithdefinedstereoscopicdepthproperties.

However,currentsolutionstostereoscopicimagecapturehavedrawbacksduetothefactthattheymapa

singlerangeofscenedepthontothetargetdisplaydepthrangeasawhole.Ifthescenedepthrangeislargerthan

theavailabledisplaydepthrangethiswillresultincompressionofperceiveddepth,asinfigure1(a).Thishas

theconsquencethataregionofinterestinscenedepthcannotbeguaranteedagoodrepresentationinperceived

depthinthedisplayedimage.

Furtherauthorinformation:(SendcorrespondencetoNickHolliman.)Email:n.s.holliman@durham.ac.uk,Telephone:+441913341761,Web:

http://www.durham.ac.uk/n.s.holliman/Copyright 2004 SPI and IS&T. This paper was published in Stereoscopic Displays and Applications XV, San Jose, California, January 󰀀2004 and is made available as an electronic reprint with permission of the SPIE and IS&T. One print or electronic copy may be made 󰀀for personal use only. Systematic or multiple reproduction, distribution to multiple locations via electronic or other means, duplication 󰀀of any material in this paper for a fee or for commerical purposes, or modification of the content of the paper are prohibited. Scene depth range

Perceived depth rangePhysical display

Virtual display

(a)Scene depth range

Perceived depth range(b)NRFR

Figure1.Stereoscopicimagecapturemapsascenedepthrangetoadisplayperceiveddepthrange:(a)Existingmethodsmapadefinedrangeofscenedepthtotheusabledisplayperceiveddepthasawhole.(b)Thenewalgorithmmapsdepthdifferentlyinthreeregions,nearregion(NR),regionofinterest(ROI)andfarregion(FR)allowingtheavailablestereoscopicdepthrangetobesplitdifferentlybetweenthethreeregions,forexample,asshowntheROImaybegiventhebeststereodepthrepresentation.

Ournewapproachtodepthmappingallowsthecontentcreatortodefinedifferentmappingsfromrealscene

depthtoperceiveddisplaydepthforthreedifferentranges;aregionofinterest(ROI)inthesceneandanear

region(NR)andfarregion(FR)inthescene,asshowninfigure1(b).Allthreeregionscanbemappedtoa

differentproportionoftheavailableperceiveddepthrangeonthetarget3Ddisplay.Theresultisthattheimage

creatorcanchoosetogivebetterstereoscopicrepresentationtotheROIcomparedtothenearandfarregions.

Weaimtoretainthesceneoutsidetheregionofinterest,ratherthanclipitordisguiseitbyblurring,because

inmanyapplicationsitisimportanttoseecontextaroundtheregionofinterest.Forexampleinagame,or

otherinteractiveapplication,itcanbeimportanttoseeintothedistancetocheckwhatiscoming,butitisnot

importanttohavethisrepresentedusinghighresolutionperceivedstereodepth,particularlywhenotherdepth

cuesareavailable.4Applicationsthatwillbenefitfromthenewalgorithmareprimarilythosethatarecomputer

graphicsbasedandincludecomputergames,medical,scientificandinformationvisualisation.

2.BACKGROUND

Thebenefitsprovidedbystereoscopicdisplaysarewellknown3,5–7andinclude;depthperceptionrelativetoa

displaysurface,perceptionofstructureinvisuallycomplexscenes,improvedperceptionofsurfacecurvature,

improvedmotionjudgementandimprovedperceptionofsurfacematerialtype.Thesebenefitshavebeento

showntobehelpfulinprovidingusersabetterunderstandingorappreciationofvisualinformationinarange

ofstudiesindifferentapplicationdomains.4,8–10

Animportantstepinobtainingthesebenefitsistousehighquality3Ddisplays,withgoodbasic2Dimage

quality(brightness,contrast,highresolution,fullcolour,movingimages)aswellasgood3Dimagequality(image

channelswellalignedspatially,inbrightnessandcontrastandwithlowinter-channelcrosstalk).3,8Suchdisplays

arebecomingavailablefordesktopandlaptopPCsystemsfromcompaniesincludingDimensionTechnologies,

Stereographics,SeeRealandSharpCorporation.3

Fortheuserthequalityofthestereoscopicimagepairisatleastasimportantasthequalityofthedisplayitis

presentedon.Theimagepairneedstobecapturedwithalignedcameras(orrectifiedtoappearasifithasbeen)

andtohaveminimaldifferencesinluminance,contrast,focusandmagnification.Syntheticcomputergraphics

systemsarebetterthanopticalphotographicsystemsinthisrespectastheimagingprocessisnotaffectedby

physicalalignmentdifferencesbetweenthetwocapturechannels.Forallstereoscopicimagecapturemethodsa

keyrequirementistocontroltheamountofperceiveddepthseenwhentheimageisdisplayedonthetarget3D

displayasthisdirectlyaffectstheviewer’sabilitytoperceiveacomfortablefused3Dimage.