Photo-inspired model-driven 3D object modeling

ACM Reference FormatXu, K., Zheng, H., Zhang, H., Cohen-Or, D., Liu, L., Xiong, Y. 2011. Photo-Inspired Model-Driven 3D Object Modeling. ACM Trans. Graph. 30, 4, Article 80 (July 2011), 10 pages. DOI = 10.1145/1964921.1964975 http://doi.acm.org/10.1145/1964921.1964975.

Copyright NoticePermission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profi t or direct commercial advantage and that copies show this notice on the fi rst page or initial screen of a display along with the full citation. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, to redistribute to lists, or to use any component of this work in other works requires prior specifi c permission and/or a fee. Permissions may be requested from Publications Dept., ACM, Inc., 2 Penn Plaza, Suite 701, New York, NY 10121-0701, fax +1 (212) 869-0481, or permissions@acm.org.© 2011 ACM 0730-0301/2011/07-ART80 $10.00 DOI 10.1145/1964921.1964975 http://doi.acm.org/10.1145/1964921.1964975Photo-InspiredModel-Driven3DObjectModeling

KaiXu∗†HanlinZheng‡HaoZhang†DanielCohen-Or§LigangLiu‡YueshanXiong†

∗NationalUniversityofDefenseTechnology†SimonFraserUniversity‡ZhejiangUniversity§Tel-AvivUniversity

Figure1:Photo-inspired3Dmodelingofachairfromfourdifferent3Dcandidates(cyan).Thenewmodels(yellow)arecreatedasgeometric

variationsofthecandidatestofitthetargetobjectinthephotowhilepreservingthe3Dstructureofthecandidates.

Abstract

Weintroduceanalgorithmfor3Dobjectmodelingwheretheuser

drawscreativeinspirationfromanobjectcapturedinasinglepho-

tograph.Ourmethodleveragestherichsourceofphotographsfor

creative3Dmodeling.However,withonlyaphotoasaguide,cre-

atinga3Dmodelfromscratchisadauntingtask.Wesupportthe

modelingprocessbyutilizinganavailablesetof3Dcandidatemod-

els.Specifically,theusercreatesadigital3Dmodelasageometric

variationfroma3Dcandidate.Ourmodelingtechniqueconsists

oftwomajorsteps.Thefirststepisauser-guidedimage-space

objectsegmentationtorevealthestructureofthephotographedob-

ject.Thecorestepisthesecondone,inwhicha3Dcandidate

isautomaticallydeformedtofitthephotographedtargetunderthe

guidanceofsilhouettecorrespondence.Thesetofcandidatemod-

elshavebeenpre-analyzedtopossessusefulhigh-levelstructural

information,whichisheavilyutilizedinbothstepstocompensate

fortheill-posednessoftheanalysisandmodelingproblemsbased

onlyoncontentinasingleimage.Equallyimportant,thestructural

informationispreservedbythegeometricvariationsothatthefinal

productiscoherentwithitsinheritedstructuralinformationreadily

usableforsubsequentmodelrefinementorprocessing.Links:DLPDFWEBVIDEO

1Introduction

Contentcreationin3Disoneofthemostfundamentaltasksincom-

putergraphics.Theultimategoalistoallowartistsandevennovice

userstoquicklyturnadesignconceptintoadigital3Dmodel.Cre-

ativityisoftencalleduponduringdesignandmodelingandassuchtheuserneedstobeinspired[ChaudhuriandKoltun2010].The

inspirationmayarisefrompureimagination,butmoreoftenthan

not,itcantraceitsoriginstooneormoreexistingconceptswith

theendproductbeingavariationorcompositionfromoneormore

existingmodels[Funkhouseretal.2004;Kraevoyetal.2007;Lee

andFunkhouser2008;ChaudhuriandKoltun2010].Moreover,the

modelingprocessdoesnotendwithaninitialcreation—thecre-

ated3Dmodelismeanttobesubsequentlyrefinedandmanipulated.

Itisthereforehighlydesirableforthecreatedmodeltobereadily

usableforsuchfurtherprocessing.

Inthispaper,weintroduceanalgorithmforcreative3Dmodel-

ingwheretheuserisinspiredbyasinglephotographandthecre-

ationprocessisdrivenbyanavailablesetof3Dcandidatemodels.

Specifically,theusercreatesarealisticandreadily-usabledigital

3Dmodelasageometricvariationfromoneofthe3Dcandidates.

Wefocusonthemodelingofman-madeobjects.Evenwithinthe

sameobjectclass,man-madeobjectsoftenexhibitimmenselyrich

shapevariability(e.g.,considerallthechairs,tables,orlampswe

encounter)whichprovidesthemodelingchallenge.

Photographsprovideperhapstherichestsourceofcreativeinspi-

ration.Theyareeasytofindandacquireandthecapturedobjects

areshownintheirnaturalappearanceandsurroundingstoprovide

themostinspiringmodelingcontextfortheuser.Requiringonlya

singlephotographinsteadofcapturesfrommultipleviewsallows

directuseofthevastsourceofimagesthatarealreadyavailable

on-lineorelsewhere.However,withonlyaphotoasaguide,cre-

atinga3Dmodelfromscratchisadauntingtask.Wesupportthe

modelingbyutilizingasetof3Dcandidates.Thecreatedmodel

isageometricvariationfromtheset,obtainedbydeformingacan-

didatemodelsothatitssilhouetteintheappropriateviewmatches

thatofthetargetobjectinthephotograph;seeFigure1.

Inoursetting,eachcandidatemodelhasbeenpre-analyzedsothat

itsgeometryrepresentationisendowedbyhigh-levelstructuralin-

formationtodrivetheobjectcreationprocess.Ononehand,the

geometryandstructureofthe3Dcandidatescaneffectivelyguide

objectanalysiswithinthephotograph.Moreimportantly,thedefor-

mationappliedtothechosencandidateisstructure-preserving—

itretainsthestructuralinformationinthecandidatesothatthepro-

ducedvariationremainscoherentandreadilyusable.Atthesame

time,thehigh-levelstructureofthecandidateisexploitedtoprovide

extraconstraintsincludingsymmetrytoalleviatetheill-posedness