A note on learning characteristic decision trees

  • 格式:pdf
  • 大小:67.20 KB
  • 文档页数:8

LearningCharacteristicDecisionTrees

PaulDavidsson

DepartmentofComputerScience,LundUniversity

Box118,S–22100Lund,Sweden

E-mail:Paul.Davidsson@dna.lth.se

Abstract

DecisiontreesconstructedbyID3-likealgorithmssufferfromaninabilityofdetectingin-stancesofcategoriesnotpresentinthesetoftrainingexamples,i.e.,theyarediscriminativerepresentations.Instead,suchinstancesareassignedtooneoftheclassesactuallypresentinthetrainingset,resultinginundesiredmisclassifications.Twomethodsofreducingthisproblembylearningcharacteristicrepresentationsarepresented.Thecentralideabehindbothmethodsistoaugmenteachleafofthedecisiontreewithasubtreecontainingaddi-tionalinformationconcerningeachfeature’svaluesinthatleaf.Thisisdonebycomputingtwolimits(lowerandupper)foreveryfeaturefromthetraininginstancesbelongingtotheleaf.Asubtreeisthenconstructedfromtheselimitsthattestseveryfeature;ifthevalueisbelowthelowerlimitorabovetheupperlimitforsomefeature,theinstancewillberejected,i.e.,regardedasbelongingtoanovelclass.Thissubtreeisthenappendedtotheleaf.Thefirstmethodpresentedcorrespondstocreatingamaximumspecificdescription,whereasthesecondisanovelmethodthatmakesuseoftheinformationaboutthestatisticaldistributionofthefeaturevaluesthatcanbeextractedfromthetrainingexamples.Animportantprop-ertyofthenovelmethodisthatthedegreeofgeneralizationcanbecontrolled.Themeth-odsareevaluatedempiricallyintwodifferentdomains,theIrisclassificationproblemandanovelcoinclassificationproblem.Itisconcludedthatthedynamicalpropertiesofthesec-ondmethodmakesitpreferableinmostapplications.Finally,wearguethatthismethodisverygeneralinthatit,inprinciple,canbeappliedtoanyempiricallearningalgorithm.

1Introduction

Oneoftheoftenignoredproblemsforalearningsystemistoknowwhenitencountersanin-

stanceofanunknowncategory.Intheoreticalcontextsthisproblemisoftenregardedasbeingof

minorimportance,mainlybecauseitisassumedthattheproblemdomainsunderstudyareclosed

(i.e.,allrelevantinformationisknowninadvanced).However,inmanypracticalapplicationsit

cannotbeassumedthateverycategoryisrepresentedinthesetoftrainingexamples(i.e.,they

areopendomains)andsometimesthecostofamisclassificationistoohigh.Whatisneededin

suchsituationsistheabilitytorejectinstancesofcategoriesthatthesystemhasnotbeentrained

on.Forexample,considerthedecisionmechanisminacoin-sortingmachineofthekindoften

usedinbankoffices.Itstaskistosortandcountalimitednumberofdifferentcoins(forinstance,

aparticularcountry’s),andtorejectallothercoins.Supposingthatthisdecisionmechanismis

tobelearned,itisforpracticalreasonsimpossibletotrainthelearningsystemoneverypossible

kindofcoin,genuineorfaked.Rather,itisdesiredthatthesystemshouldbetrainedonlyon

thekindsofcoinsitissupposedtoaccept.Anotherexamplearedecisionsupportsystems,forfeature2

feature1

Atleast,intheoriginalversionsofthesealgorithms.2TheMethods

Bothmethodsarebasedontheideaofaugmentingeachleafofthetreeresultingfromadecision

treealgorithmwithasubtree.Thepurposeofthesesubtreesistoimposefurtherrestrictionson

thefeaturevalues.Alowerandaupperlimitarecomputedforeveryfeature.Thesewillserve

astests:ifthefeaturevalueoftheinstancetobeclassifiedisbelowthelowerlimitorabove

theupperlimitforoneormoreofthefeatures,theinstancewillberejected,i.e.,regardedas

belongingtoanovelclass,otherwiseitwillbeclassifiedaccordingtotheoriginaldecisiontree.

Thus,whenanewinstanceistobeclassified,thedecisiontreeisfirstappliedasusual,andthen,

whenaleafwouldhavebeenreached,everyfeatureoftheinstanceischeckedtoseeifitbelongs

totheintervaldefinedbythelowerandtheupperlimit.Ifallfeaturesofthenewinstanceare

insidetheirintervaltheclassificationisstillvalid,otherwisetheinstancewillberejected.

Inthefirstmethodwecomputetheminimumandmaximumfeaturevaluefromthetraining

instancesoftheleafandletthesebethelowerandupperlimitsrespectively.Thisapproachwill

yieldamaximumspecificdescription(cf.themodificationofCN2byHolteetal.[3]).

Whilebeingintuitiveandstraight-forward,thismethodisalsoratherstaticinthesensethat

thereisnowayofcontrollingthevaluesofthelimits,i.e.,thedegreeofgeneralization.Such

anabilitywouldbedesirable,forexample,whensomeinstancesthatwouldhavebeencorrectly

classifiedbytheoriginaldecisiontreearerejectedbytheaugmentedtree(whichhappensifany

ofitsfeaturevaluesisonthewrongsideofalimit).Actually,thereisatrade-offbetweenthe

numberoffailuresofthiskindandthenumberofmisclassifiedinstances.Howitshouldbe

balancedis,ofcourse,dependentoftheapplication(i.e.,thecostsofmisclassificationandre-

jection).Sinceitisimpossibleintheabovemethodtobalancethistrade-off,amoredynamic

methodinwhichitcanbecontrolledhasbeendeveloped.

Thecentralideaofthisnovelmethodistomakeuseofstatisticalinformationconcerning

thedistributionofthefeaturevaluesoftheinstancesinaleaf.Foreveryfeaturewecomputethe