频繁模式挖掘中Apriori、FP-Growth和Eclat算法的实现和对⽐（Python实现）最近上数据挖掘的课程，其中学习到了频繁模式挖掘这⼀章，这章介绍了三种算法，Apriori、FP-Growth和Eclat算法；由于对于不同的数据来说，这三种算法的表现不同，所以我们本次就对这三种算法在不同情况下的效率进⾏对⽐。
从⽽得出适合相应算法的情况。
GitHub：（⼀）算法原理其中相应的算法原理在之前的博客中都有⾮常详细的介绍，这⾥就不再赘述，这⾥给出三种算法⼤概的介绍但是这⾥给出每个算法的关键点：1.1 Apriori算法：限制候选产⽣发现频繁项集重要性质：频繁项集所有⾮空⼦集也⼀定是频繁的。
主要步骤：1. 连接2. 剪枝特点：需要多次扫描数据库，对于⼤规模数据效率很低！1.2 FP-Growth算法通过模式增长挖掘频繁模式主要步骤：1. 构建频繁模式树2. 构造条件模式基3. 挖掘频繁模式特点：两次扫描数据库，采⽤分治的策略有效降低搜索开销1.3 Eclat算法使⽤垂直格式挖掘频繁项集主要步骤：1. 将数据倒排{ item：TID_set }2. 通过求频繁k项集的交集来获取k+1项集特点：仅需要⼀次扫描数据库，TID集合很长的话需要消耗⼤量的内存和计算时间（⼆）算法实现由于各个博客给出的算法实现并不统⼀，⽽且本⼈在实现《机器学习实战》中FP-Growth算法的时候发现，在在创建FP-Tree时根据headTable中元素的⽀持度顺序的排序过程中，这个地⽅的排序⽅法写的有问题，当在模式稠密时，具有很多⽀持度相同的项集，书中的代码并没有考虑着⼀点，所以如果遇到⽀持度相同的项集那个就会出现⼀定的随机性，导致建树过程出错，最后的频繁项集结果会偏⼩，因此这⾥对改错误进⾏了纠正，在⽀持度相同时，添加了按照项集排序的规则，这样建⽴的FP-Tree才完全正确。
1.1 Apriori算法实现：1# -*- coding: utf-8 -*-2'''3@author: Infaraway4@time: 2017/4/15 12:545@Function:6'''789def init_c1(data_set_dict, min_support):10 c1 = []11 freq_dic = {}12for trans in data_set_dict:13for item in trans:14 freq_dic[item] = freq_dic.get(item, 0) + data_set_dict[trans]15# 优化初始的集合，使不满⾜最⼩⽀持度的直接排除16 c1 = [[k] for (k, v) in freq_dic.iteritems() if v >= min_support]17 c1.sort()18return map(frozenset, c1)192021def scan_data(data_set, ck, min_support, freq_items):22"""23计算Ck中的项在数据集合中的⽀持度，剪枝过程24 :param data_set:25 :param ck:26 :param min_support: 最⼩⽀持度27 :param freq_items: 存储满⾜⽀持度的频繁项集28 :return:29"""30 ss_cnt = {}31# 每次遍历全体数据集32for trans in data_set:33for item in ck:34# 对每⼀个候选项集，检查是否是 term中的⼀部分（⼦集），即候选项能否得到⽀持35if item.issubset(trans):36 ss_cnt[item] = ss_cnt.get(item, 0) + 137 ret_list = []38for key in ss_cnt:39 support = ss_cnt[key] # 每个项的⽀持度40if support >= min_support:41 ret_list.insert(0, key) # 将满⾜最⼩⽀持度的项存⼊集合42 freq_items[key] = support #43return ret_list444546def apriori_gen(lk, k):47"""48由Lk的频繁项集⽣成新的候选项集连接过程49 :param lk: 频繁项集集合50 :param k: k 表⽰集合中所含的元素个数51 :return: 候选项集集合52"""53 ret_list = []54for i in range(len(lk)):55for j in range(i+1, len(lk)):56 l1 = list(lk[i])[:k-2]57 l2 = list(lk[j])[:k-2]58 l1.sort()59 l2.sort()60if l1 == l2:61 ret_list.append(lk[i] | lk[j]) # 求并集62# retList.sort()63return ret_list646566def apriori_zc(data_set, data_set_dict, min_support=5):67"""68 Apriori算法过程69 :param data_set: 数据集70 :param min_support: 最⼩⽀持度，默认值 0.571 :return:72"""73 c1 = init_c1(data_set_dict, min_support)74 data = map(set, data_set) # 将dataSet集合化，以满⾜scanD的格式要求75 freq_items = {}76 l1 = scan_data(data, c1, min_support, freq_items) # 构建初始的频繁项集77 l = [l1]78# 最初的L1中的每个项集含有⼀个元素，新⽣成的项集应该含有2个元素，所以 k=279 k = 280while len(l[k - 2]) > 0:81 ck = apriori_gen(l[k - 2], k)82 lk = scan_data(data, ck, min_support, freq_items)83 l.append(lk)84 k += 1 # 新⽣成的项集中的元素个数应不断增加85return freq_itemsView Code1.2 FP-Growth算法实现：1）FP_Growth⽂件：在create_tree()函数中修改《机器学习实战》中的代码：############################################################################################## # 这⾥修改机器学习实战中的排序代码：ordered_items = [v[0] for v in sorted(local_data.items(), key=lambda kv: (-kv[1], kv[0]))]##############################################################################################1# -*- coding: utf-8 -*-2"""3@author: Infaraway4@time: 2017/4/15 16:075@Function:6"""7from DataMining.Unit6_FrequentPattern.FP_Growth.TreeNode import treeNode8910def create_tree(data_set, min_support=1):11"""12创建FP树13 :param data_set: 数据集14 :param min_support: 最⼩⽀持度15 :return:16"""17 freq_items = {} # 频繁项集18for trans in data_set: # 第⼀次遍历数据集19for item in trans:20 freq_items[item] = freq_items.get(item, 0) + data_set[trans]2122 header_table = {k: v for (k, v) in freq_items.iteritems() if v >= min_support} # 创建头指针表23# for key in header_table:24# print key, header_table[key]2526# ⽆频繁项集27if len(header_table) == 0:28return None, None29for k in header_table:30 header_table[k] = [header_table[k], None] # 添加头指针表指向树中的数据31# 创建树过程32 ret_tree = treeNode('Null Set', 1, None) # 根节点3334# 第⼆次遍历数据集35for trans, count in data_set.items():36 local_data = {}37for item in trans:38if header_table.get(item, 0):39 local_data[item] = header_table[item][0]40if len(local_data) > 0:41############################################################################################## 42# 这⾥修改机器学习实战中的排序代码：43 ordered_items = [v[0] for v in sorted(local_data.items(), key=lambda kv: (-kv[1], kv[0]))]44############################################################################################## 45 update_tree(ordered_items, ret_tree, header_table, count) # populate tree with ordered freq itemset46return ret_tree, header_table474849def update_tree(items, in_tree, header_table, count):50'''51 :param items: 元素项52 :param in_tree: 检查当前节点53 :param header_table:54 :param count:55 :return:56'''57if items[0] in in_tree.children: # check if ordered_items[0] in ret_tree.children58 in_tree.children[items[0]].increase(count) # incrament count59else: # add items[0] to in_tree.children60 in_tree.children[items[0]] = treeNode(items[0], count, in_tree)61if header_table[items[0]][1] is None: # update header table62 header_table[items[0]][1] = in_tree.children[items[0]]63else:64 update_header(header_table[items[0]][1], in_tree.children[items[0]])65if len(items) > 1: # call update_tree() with remaining ordered items66 update_tree(items[1::], in_tree.children[items[0]], header_table, count)676869def update_header(node_test, target_node):70'''71 :param node_test:72 :param target_node:73 :return:74'''75while node_test.node_link is not None: # Do not use recursion to traverse a linked list!76 node_test = node_test.node_link77 node_test.node_link = target_node787980def ascend_tree(leaf_node, pre_fix_path):81'''82遍历⽗节点，找到路径83 :param leaf_node:84 :param pre_fix_path:85 :return:86'''87if leaf_node.parent is not None:88 pre_fix_path.append(leaf_)89 ascend_tree(leaf_node.parent, pre_fix_path)909192def find_pre_fix_path(base_pat, tree_node):93'''94创建前缀路径95 :param base_pat: 频繁项96 :param treeNode: FP树中对应的第⼀个节点97 :return:98'''99# 条件模式基100 cond_pats = {}101while tree_node is not None:102 pre_fix_path = []103 ascend_tree(tree_node, pre_fix_path)104if len(pre_fix_path) > 1:105 cond_pats[frozenset(pre_fix_path[1:])] = tree_node.count106 tree_node = tree_node.node_link107return cond_pats108109110def mine_tree(in_tree, header_table, min_support, pre_fix, freq_items):111'''112挖掘频繁项集113 :param in_tree:114 :param header_table:115 :param min_support:116 :param pre_fix:117 :param freq_items:118 :return:119'''120# 从⼩到⼤排列table中的元素，为遍历寻找频繁集合使⽤121 bigL = [v[0] for v in sorted(header_table.items(), key=lambda p: p[1])] # (sort header table) 122for base_pat in bigL: # start from bottom of header table123 new_freq_set = pre_fix.copy()124 new_freq_set.add(base_pat)125# print 'finalFrequent Item: ',new_freq_set #append to set126if len(new_freq_set) > 0:127 freq_items[frozenset(new_freq_set)] = header_table[base_pat][0]128 cond_patt_bases = find_pre_fix_path(base_pat, header_table[base_pat][1])129 my_cond_tree, my_head = create_tree(cond_patt_bases, min_support)130# print 'head from conditional tree: ', my_head131if my_head is not None: # 3. mine cond. FP-tree132# print 'conditional tree for: ',new_freq_set133# my_cond_tree.disp(1)134 mine_tree(my_cond_tree, my_head, min_support, new_freq_set, freq_items)135136137def fp_growth(data_set, min_support=1):138 my_fp_tree, my_header_tab = create_tree(data_set, min_support)139# my_fp_tree.disp()140 freq_items = {}141 mine_tree(my_fp_tree, my_header_tab, min_support, set([]), freq_items)142return freq_itemsView Code2）treeNode对象⽂件1# -*- coding: utf-8 -*-2'''3@author: Infaraway4@time: 2017/3/31 0:145@Function:6'''789class treeNode:10def__init__(self, name_value, num_occur, parent_node):11 = name_value # 节点元素名称12 self.count = num_occur # 出现的次数13 self.node_link = None # 指向下⼀个相似节点的指针，默认为None14 self.parent = parent_node # 指向⽗节点的指针15 self.children = {} # 指向孩⼦节点的字典⼦节点的元素名称为键，指向⼦节点的指针为值1617def increase(self, num_occur):18"""19增加节点的出现次数20 :param num_occur: 增加数量21 :return:22"""23 self.count += num_occur2425def disp(self, ind=1):26print'' * ind, , '', self.count27for child in self.children.values():28 child.disp(ind + 1)View Code1.3 Eclat算法实现1# -*- coding: utf-8 -*-2"""3@author: Infaraway4@time: 2017/4/15 19:335@Function:6"""78import sys9import time10 type = sys.getfilesystemencoding()111213def eclat(prefix, items, min_support, freq_items):14while items:15# 初始遍历单个的元素是否是频繁16 key, item = items.pop()17 key_support = len(item)18if key_support >= min_support:19# print frozenset(sorted(prefix+[key]))20 freq_items[frozenset(sorted(prefix+[key]))] = key_support21 suffix = [] # 存储当前长度的项集22for other_key, other_item in items:23 new_item = item & other_item # 求和其他集合求交集24if len(new_item) >= min_support:25 suffix.append((other_key, new_item))26 eclat(prefix+[key], sorted(suffix, key=lambda item: len(item[1]), reverse=True), min_support, freq_items)27return freq_items282930def eclat_zc(data_set, min_support=1):31"""32 Eclat⽅法33 :param data_set:34 :param min_support:35 :return:36"""37# 将数据倒排38 data = {}39 trans_num = 040for trans in data_set:41 trans_num += 142for item in trans:43if item not in data:44 data[item] = set()45 data[item].add(trans_num)46 freq_items = {}47 freq_items = eclat([], sorted(data.items(), key=lambda item: len(item[1]), reverse=True), min_support, freq_items)48return freq_itemsView Code（三）试验阶段：这样我们就统⼀了三种算法的调⽤以及返回值，现在我们可以开始试验阶段了，我们在试验阶段分别根据最⼩⽀持度阈值和数据规模的变化来判断这三种算法的效率：⾸先我们先统⼀调⽤者三个算法：1def test_fp_growth(minSup, dataSetDict, dataSet):2 freqItems = fp_growth(dataSetDict, minSup)3 freqItems = sorted(freqItems.iteritems(), key=lambda item: item[1])4return freqItems567def test_apriori(minSup, dataSetDict, dataSet):8 freqItems = apriori_zc(dataSet, dataSetDict, minSup)9 freqItems = sorted(freqItems.iteritems(), key=lambda item: item[1])10return freqItems111213def test_eclat(minSup, dataSetDict, dataSet):14 freqItems = eclat_zc(dataSet, minSup)15 freqItems = sorted(freqItems.iteritems(), key=lambda item: item[1])16return freqItems然后实现数据规模变化的效率改变1def do_experiment_min_support():23 data_name = 'unixData8_pro.txt'4 x_name = "Min_Support"5 data_num = 15006 minSup = data_num / 678 dataSetDict, dataSet = loadDblpData(open("dataSet/" + data_name), ',', data_num)9 step = minSup / 5 # #################################################################10 all_time = []11 x_value = []12for k in range(5):1314 x_value.append(minSup) # ################################################################# 15if minSup < 0: # #################################################################16break17 time_fp = 018 time_et = 019 time_ap = 020 freqItems_fp = {}21 freqItems_eclat = {}22 freqItems_ap = {}23for i in range(10):24 ticks0 = time.time()25 freqItems_fp = test_fp_growth(minSup, dataSetDict, dataSet)26 time_fp += time.time() - ticks027 ticks0 = time.time()28 freqItems_eclat = test_eclat(minSup, dataSetDict, dataSet)29 time_et += time.time() - ticks030 ticks0 = time.time()31 freqItems_ap = test_apriori(minSup, dataSetDict, dataSet)32 time_ap += time.time() - ticks033print"minSup :", minSup, " data_num :", data_num, \34" freqItems_fp:", len(freqItems_fp), " freqItems_eclat:", len(freqItems_eclat), " freqItems_ap:", len(35 freqItems_ap)36print"fp_growth:", time_fp / 10, " eclat:", time_et / 10, " apriori:", time_ap / 1037# print_freqItems("show", freqItems_eclat)38 minSup -= step # #################################################################39 use_time = [time_fp / 10, time_et / 10, time_ap / 10]40 all_time.append(use_time)41# print use_time42 y_value = []43for i in range(len(all_time[0])):44 tmp = []45for j in range(len(all_time)):46 tmp.append(all_time[j][i])47 y_value.append(tmp)48 plot_pic(x_value, y_value, data_name, x_name)49return x_value, y_valueView Code然后实现最⼩⽀持度变化的效率改变1def do_experiment_data_size():23 data_name = 'kosarakt.txt'4 x_name = "Data_Size"5 data_num = 20000067 step = data_num / 5 # #################################################################8 all_time = []9 x_value = []10for k in range(5):11 minSup = data_num * 0.01012 dataSetDict, dataSet = loadDblpData(open("dataSet/"+data_name), '', data_num)13 x_value.append(data_num) # #################################################################14if data_num < 0: # #################################################################15break16 time_fp = 017 time_et = 018 time_ap = 019 freqItems_fp = {}20 freqItems_eclat = {}21 freqItems_ap = {}22for i in range(2):23 ticks0 = time.time()24 freqItems_fp = test_fp_growth(minSup, dataSetDict, dataSet)25 time_fp += time.time() - ticks026 ticks0 = time.time()27 freqItems_eclat = test_eclat(minSup, dataSetDict, dataSet)28 time_et += time.time() - ticks029 ticks0 = time.time()30# freqItems_ap = test_apriori(minSup, dataSetDict, dataSet)31# time_ap += time.time() - ticks032print"minSup :", minSup, " data_num :", data_num, \33" freqItems_fp:", len(freqItems_fp), " freqItems_eclat:", len(freqItems_eclat), " freqItems_ap:", len(freqItems_ap) 34print"fp_growth:", time_fp / 10, " eclat:", time_et / 10, " apriori:", time_ap / 1035# print_freqItems("show", freqItems_eclat)36 data_num -= step # #################################################################37 use_time = [time_fp / 10, time_et / 10, time_ap / 10]38 all_time.append(use_time)39# print use_time4041 y_value = []42for i in range(len(all_time[0])):43 tmp = []44for j in range(len(all_time)):45 tmp.append(all_time[j][i])46 y_value.append(tmp)47 plot_pic(x_value, y_value, data_name, x_name)48return x_value, y_valueView Code同时为了观察⽅便，我们需要对三种算法返回的结果进⾏绘图1# -*- coding: utf-8 -*-2"""3@author: Infaraway4@time: 2017/4/16 20:485@Function:6"""78import matplotlib.pyplot as plt91011def plot_pic(x_value, y_value, title, x_name):12 plot1 = plt.plot(x_value, y_value[0], 'r', label='Kulc') # use pylab to plot x and y13 plot2 = plt.plot(x_value, y_value[1], 'g', label='IR') # use pylab to plot x and y14# plot3 = plt.plot(x_value, y_value[2], 'b', label='Apriori') # use pylab to plot x and y15 plt.title(title) # give plot a title16 plt.xlabel(x_name) # make axis labels17 plt.ylabel('value ')18 plt.legend(loc='upper right') # make legend1920 plt.show() # show the plot on the screenView Code将两个部分统⼀执⾏：1if__name__ == '__main__':23# x_value, y_value = do_experiment_min_support()4# x_value, y_value = do_experiment_data_size()5# do_test()（四）实验结果分析：本次实验我们主要从以下⼏个⽅⾯来讨论三种算法的效率：数据规模⼤⼩最⼩⽀持度阈值长事物数据模式的稠密性4.1 数据规模⼤⼩：数据集：unxiData8规模：900-1500Min_support = 1/30时 Min_support = 1/20时数据集：kosarakt规模：6000-10000Min_support = 1/50 Min_support = 1/80 Min_support = 1/100结论：⼀般情况下，数据规模越⼤，使⽤Apriori算法的效率越低，因为该算法需要多次扫描数据库，当数据量越⼤时，扫描数据库带来的消耗越多。
