A note on learning characteristic decision trees
- 格式:pdf
- 大小:67.20 KB
- 文档页数:8
LearningCharacteristicDecisionTrees
PaulDavidsson
DepartmentofComputerScience,LundUniversity
Box118,S–22100Lund,Sweden
E-mail:Paul.Davidsson@dna.lth.se
Abstract
DecisiontreesconstructedbyID3-likealgorithmssufferfromaninabilityofdetectingin-stancesofcategoriesnotpresentinthesetoftrainingexamples,i.e.,theyarediscriminativerepresentations.Instead,suchinstancesareassignedtooneoftheclassesactuallypresentinthetrainingset,resultinginundesiredmisclassifications.Twomethodsofreducingthisproblembylearningcharacteristicrepresentationsarepresented.Thecentralideabehindbothmethodsistoaugmenteachleafofthedecisiontreewithasubtreecontainingaddi-tionalinformationconcerningeachfeature’svaluesinthatleaf.Thisisdonebycomputingtwolimits(lowerandupper)foreveryfeaturefromthetraininginstancesbelongingtotheleaf.Asubtreeisthenconstructedfromtheselimitsthattestseveryfeature;ifthevalueisbelowthelowerlimitorabovetheupperlimitforsomefeature,theinstancewillberejected,i.e.,regardedasbelongingtoanovelclass.Thissubtreeisthenappendedtotheleaf.Thefirstmethodpresentedcorrespondstocreatingamaximumspecificdescription,whereasthesecondisanovelmethodthatmakesuseoftheinformationaboutthestatisticaldistributionofthefeaturevaluesthatcanbeextractedfromthetrainingexamples.Animportantprop-ertyofthenovelmethodisthatthedegreeofgeneralizationcanbecontrolled.Themeth-odsareevaluatedempiricallyintwodifferentdomains,theIrisclassificationproblemandanovelcoinclassificationproblem.Itisconcludedthatthedynamicalpropertiesofthesec-ondmethodmakesitpreferableinmostapplications.Finally,wearguethatthismethodisverygeneralinthatit,inprinciple,canbeappliedtoanyempiricallearningalgorithm.
1Introduction
Oneoftheoftenignoredproblemsforalearningsystemistoknowwhenitencountersanin-
stanceofanunknowncategory.Intheoreticalcontextsthisproblemisoftenregardedasbeingof
minorimportance,mainlybecauseitisassumedthattheproblemdomainsunderstudyareclosed
(i.e.,allrelevantinformationisknowninadvanced).However,inmanypracticalapplicationsit
cannotbeassumedthateverycategoryisrepresentedinthesetoftrainingexamples(i.e.,they
areopendomains)andsometimesthecostofamisclassificationistoohigh.Whatisneededin
suchsituationsistheabilitytorejectinstancesofcategoriesthatthesystemhasnotbeentrained
on.Forexample,considerthedecisionmechanisminacoin-sortingmachineofthekindoften
usedinbankoffices.Itstaskistosortandcountalimitednumberofdifferentcoins(forinstance,
aparticularcountry’s),andtorejectallothercoins.Supposingthatthisdecisionmechanismis
tobelearned,itisforpracticalreasonsimpossibletotrainthelearningsystemoneverypossible
kindofcoin,genuineorfaked.Rather,itisdesiredthatthesystemshouldbetrainedonlyon
thekindsofcoinsitissupposedtoaccept.Anotherexamplearedecisionsupportsystems,forfeature2
feature1
Atleast,intheoriginalversionsofthesealgorithms.2TheMethods
Bothmethodsarebasedontheideaofaugmentingeachleafofthetreeresultingfromadecision
treealgorithmwithasubtree.Thepurposeofthesesubtreesistoimposefurtherrestrictionson
thefeaturevalues.Alowerandaupperlimitarecomputedforeveryfeature.Thesewillserve
astests:ifthefeaturevalueoftheinstancetobeclassifiedisbelowthelowerlimitorabove
theupperlimitforoneormoreofthefeatures,theinstancewillberejected,i.e.,regardedas
belongingtoanovelclass,otherwiseitwillbeclassifiedaccordingtotheoriginaldecisiontree.
Thus,whenanewinstanceistobeclassified,thedecisiontreeisfirstappliedasusual,andthen,
whenaleafwouldhavebeenreached,everyfeatureoftheinstanceischeckedtoseeifitbelongs
totheintervaldefinedbythelowerandtheupperlimit.Ifallfeaturesofthenewinstanceare
insidetheirintervaltheclassificationisstillvalid,otherwisetheinstancewillberejected.
Inthefirstmethodwecomputetheminimumandmaximumfeaturevaluefromthetraining
instancesoftheleafandletthesebethelowerandupperlimitsrespectively.Thisapproachwill
yieldamaximumspecificdescription(cf.themodificationofCN2byHolteetal.[3]).
Whilebeingintuitiveandstraight-forward,thismethodisalsoratherstaticinthesensethat
thereisnowayofcontrollingthevaluesofthelimits,i.e.,thedegreeofgeneralization.Such
anabilitywouldbedesirable,forexample,whensomeinstancesthatwouldhavebeencorrectly
classifiedbytheoriginaldecisiontreearerejectedbytheaugmentedtree(whichhappensifany
ofitsfeaturevaluesisonthewrongsideofalimit).Actually,thereisatrade-offbetweenthe
numberoffailuresofthiskindandthenumberofmisclassifiedinstances.Howitshouldbe
balancedis,ofcourse,dependentoftheapplication(i.e.,thecostsofmisclassificationandre-
jection).Sinceitisimpossibleintheabovemethodtobalancethistrade-off,amoredynamic
methodinwhichitcanbecontrolledhasbeendeveloped.
Thecentralideaofthisnovelmethodistomakeuseofstatisticalinformationconcerning
thedistributionofthefeaturevaluesoftheinstancesinaleaf.Foreveryfeaturewecomputethe