DigitalVideoTranscoding
JUNXIN,MEMBER,IEEE,CHIA-WENLIN
,SENIORMEMBER,IEEE
,
AND
MING-TINGSUN
,FELLOW,IEEE
InvitedPaper
Videotranscoding,duetoitshighpracticalvaluesforawide
rangeofnetworkedvideoapplications,hasbecomeanactive
researchtopic.Inthispaper,weoutlinethetechnicalissuesand
researchresultsrelatedtovideotranscoding.Wealsodiscusstech-
niquesforreducingthecomplexity,andtechniquesforimproving
thevideoquality,byexploitingtheinformationextractedfromthe
inputvideobitstream.
Keywords—Errorresilience,motionestimation,ratecontrol,
transcodingarchitecture,videotranscoding.
I.I
NTRODUCTION
Videotranscodingistheoperationofconvertingavideo
fromoneformatintoanotherformat.Aformatisdefinedby
suchcharacteristicsasthebitrate,framerate,spatialresolu-
tion,codingsyntax,andcontent,asshowninFig.1.
Oneoftheearliestapplicationsoftranscodingistoadapt
thebitrateofaprecompressedvideostreamtoachannel
bandwidth.Forexample,aTVprogrammaybeoriginally
compressedatahighbitrateforstudioapplications,butlater
needstobetransmittedoverachannelatamuchlowerbit
rate.
Inuniversalmultimediaaccess[1],differentterminals
mayhavedifferentaccessestotheInternet,including
localaccessnetwork(LAN),digitalsubscriberline(DSL),
cable,wirelessnetworks,integratedservicesdigitalnetwork
(ISDN),anddial-up.Thedifferentaccessnetworkshave
differentchannelcharacteristicssuchasbandwidths,bit
errorrates,andpacketlossrates.Attheusers’end,network
appliancesincludinghandheldcomputers,personaldigital
assistants(PDAs),set-topboxes,andsmartcellularphones
areslatedtoreplacepersonalcomputersasthedominant
ManuscriptreceivedJanuary16,2004;revisedJuly9,2004.
J.XiniswithMitsubishiElectricResearchLaboratories,Cambridge,MA
02139USA(e-mail:jxin@merl.com).
C.-W.LiniswiththeDepartmentofComputerScienceandInformation
Engineering,NationalChungChengUniversity,Chiayi612,Taiwan,R.O.C.
(e-mail:cwlin@cs.ccu.edu.tw).
M.-T.SuniswiththeDepartmentofElectricalEngineering,University
ofWashington,Seattle,WA98195USA(e-mail:sun@ee.washington.edu).
DigitalObjectIdentifier10.1109/JPROC.2004.839620
terminalsforaccessingtheInternet.Thesenetworkter-
minalsvarysignificantlyinresourcessuchascomputing
poweranddisplaycapability.Toflexiblydelivermultimedia
datatouserswithdifferentavailableresources,accessnet-
works,andinterests,themultimediacontentmayneedto
beadapteddynamicallyaccordingtotheusageenvironment
[2].Transcodingisoneofthekeytechnologiestofulfill
thischallengingtask.Transcodingisalsousefulforcontent
adaptationforpeer-to-peernetworkingoversharedmultihop
communicationlinks[3].
Therearemanyothertranscodingapplicationsbesides
theuniversalmultimediaaccess.Instatisticalmultiplexing
[4],multiplevariable-bit-ratevideostreamsaremultiplexed
togethertoachievethestatisticalmultiplexinggain.When
theaggregatedbitrateexceedsthechannelbandwidth,a
transcodercanbeusedtoadaptthebitratesofthevideo
streamstoensurethattheaggregatedbitratealwayssat-
isfiesthechannelbandwidthconstraint.Atranscodercan
alsobeusedtoinsertnewinformationincludingcompany
logos,watermarks,aswellaserror-resiliencefeaturesinto
acompressedvideostream.Transcodingtechniquesare
alsoshownusefulforsupportingVCRtrickmodes,i.e.,
fastforward,reverseplay,etc.,foron-demandvideoap-
plications[8]–[10].Inaddition,object-basedtranscoding
techniquesarediscussedin[7]foradaptivevideocontent
delivery.Ageneralutility-basedframeworkisintroducedin
[11]toformulatesometranscodingandadaptationissues
asresource-constrainedutilitymaximizationproblems.In
[12],autility-functionpredictionisperformedusingauto-
maticfeatureextractionandregressionforMPEG-4video
transcoding.Severalrate-distortionmodelsfortranscoding
optimizationareintroducedin[13]tofacilitatetheselection
oftranscodingmethodsunderarateconstraint.Envisioning
theneedoftranscoding,theemergingMPEG-7standard[5],
whichstandardizesaframeworkfordescribingaudiovisual
contents,hasdefined“transcodinghints”tofacilitatethe
transcodingofcompressedvideocontents[6],[7].
Dynamicchangeofcodingparameterssuchasbitrates,
framerates,andspatialresolutionscouldalsobeachieved
0018-9219/$20.00©2005IEEE
84PROCEEDINGSOFTHEIEEE,VOL.93,NO.1,JANUARY2005
Fig.1.Formatconversionusingavideotranscoder.
Fig.2.Blockdiagramofastandardvideoencoder.
toalimitedextentbyscalablecoding[14],[15].How-ever,inthecurrentscalablevideocodingstandards,theenhancementlayersaregeneratedbycodingthepredictionresidualsbetweentheoriginalvideoandthebase-layervideo.Inmanyapplications,thenetworkbandwidthmayfluctuatewildlywithtime.Therefore,itmaybedifficulttosetthebase-layerbitrate.Ifthebase-layerbitrateissetlow,thebase-layervideoqualitywillberelativelylowandtheoverallvideoqualitydegradationmaybesevere,sincethepredictionbecomeslesseffective.Ontheotherhand,ifthebase-layerbitrateissethigh,thebase-layervideomaynotgetthroughthenetworkcompletely.Ingeneral,theachievablequalityofscalablecodingissignificantlylowerthanthatofnonscalablecoding.Inaddition,scalablevideocodingdemandsadditionalcomplexitiesatbothencodersanddecoders.Theinherentweaknessesofscalablecodinghavekeptitfrombeingwidelydeployedinpracticalapplica-tions.Nevertheless,scalablecodingisstillanactiveresearcharea.Anewcodingstandardisbeingdevelopedtoovercomethesedrawbacks[16].Withtheseproblemsaddressed,thescalablecodingschemesmaybemoresuitableforstreamingvideoapplicationswhenalargenumberofusersrequiredifferentlevelsofformatadaptations,sincelesscomputationisinvolved.Inthispaper,theinputtothetranscoderisacompressedvideoproducedbyastandardvideoencoder.Currentmajorvideocodingstandards—MPEG-1[17],MPEG-2[18],MPEG-4[19],H.263[20],andtheemergingH.264[21]allusehybriddiscretecosinetransform(DCT)andblock-basedmotioncompensation(MC)schemes.NotethatH.264usesanintegertransformthatapproximatesDCT.Ablockdia-gramofthestandardvideoencodersisshowninFig.2.MCreducesthetemporalredundancy.DCTreducesthespatial
redundancyandachievesenergycompaction.Quantization
isperformedtoachievehighercompressionratio.Vari-
able-lengthcoding(VLC)isappliedafterthequantization
toreducetheremainingredundancy.Adecoderisembedded
intheencodertoreconstructvideoframes,whicharestored
intheframememoryforpredictionoffutureframes.
Astraightforwardrealizationofatranscodertocascade
adecoderandanencoder:thedecoderdecodesthecom-
pressedinputvideoandtheencoderreencodesthedecoded
videointothetargetformat.Itiscomputationallyveryex-
pensive.Therefore,reducingthecomplexityofthestraight-
forwarddecoder-encoderimplementationisamajordriving
forcebehindmanyresearchactivitiesontranscoding.
Whatmakestranscodingdifferentfromvideoencodingis
thatthetranscodinghasaccesstomanycodingparameters
andstatisticsthatcanbeeasilyobtainedfromtheinputcom-
pressedvideostream.Theymaybeusednotonlytosim-
plifythecomputation,butalsotoimprovethevideoquality.
Thetranscodingcanbeconsideredasaspecialtwo-passen-
coding:the“first-pass”encodingproducestheinputcom-
pressedvideostream,andthe“second-pass”encodinginthe
transcodercanusetheinformationobtainedfromthefirst-
passtodoabetterencoding.Therefore,itispossibleforthe
transcodertoachievebettervideoqualitythanthestraightfor-
wardimplementation,wheretheencodingissinglepass.The
challengeoftheresearchontranscodingisthenhowtointel-
ligentlyutilizethecodingstatisticsandparametersextracted
fromtheinputtoachievethebestpossiblevideoqualityand
thelowestpossiblecomputationalcomplexity.
Inthispaper,wediscusstheissuesandresearchresultsre-
latedtothetranscodingofvideostreamscompressedusing
XINetal.:DIGITALVIDEOTRANSCODING85
