仿真数据论文：仿真数据并行分布式挖掘算法研究.docVIP

仿真数据论文：仿真数据并行分布式挖掘算法研究 【中文摘要】仿真实验产生了大规模的仿真数据,为了从中提取有用的信息和知识,更好地认识和改进系统,帮助决策人员分析决策,可以用数据挖掘方法来进行仿真数据分析。由于仿真数据具有维度高、规模大的特点,需要采用并行挖掘算法来提高效率。又根据仿真数据分布式存储的特点,避免大规模数据集中所带来的开销以及安全性,需要研究分布式挖掘算法。本文主要的研究工作包括以下内容:根据仿真数据固有特点和分布式存储特点,提出了并行分布式挖掘的需求。根据数据挖掘的基本流程,总结了仿真数据挖掘的一般流程。寻规律和寻优是仿真中的两种重要需求,相应地研究了两种常用的挖掘方法:关联规则和决策树,来挖掘系统中的一些关联规律,以及对仿真系统的优化。对于仿真实验寻规律的需求,将关联规则挖掘方法应用其中。对基本的Apriori算法进行研究改进,采用了向量化的数据结构,减少了输入数据集的存储空间,可以将数据集放入内存,避免对数据库多次扫描带来的I/O开销,提高了算法的效率。用vector容器替代哈希树存储候选项集,减少了算法的空间复杂度。同时为了适应仿真数据大规模的特点,根据CD(Count Distribute)算法的思想对算法进行并行化,并且对算法的拓展性进行了实验设计分析。针对解决仿真实验寻优的需求,采用了决策树挖掘方法。由于仿真数据分布式存储的特点,研究了基于元学习的分布式分类器,以及决策树的两种并行化方法,同步树构建方法和分割树构建方法,实现基于ID3算法的同步树决策树挖掘算法,最后通过仿真测试验证了算法的有效性。 【英文摘要】Simulation experiments generate massive simulation data, in order to extract useful information and knowledge, understand and improve the system better, help the deciders make decisions, can use data mining method in simulation data analysis. Because the simulation data has high dimensions and big scale, parallel mining algorithm is needed to improve efficiency. And also these data is stored in distribute place, centralizing the data will be very expansive and may be not safe, distributed mining algorithm should be considered. The context of this paper is given as fallow.Due to the instant and distribute attribute of simulation data, the need of parallel and distributed data mining is proposed. According to the basic process of data mining, we conclude the common process of mining simulation data. Find the rules and optimism is two important needs in simulation, there are two favorite mining method correspondingly, association rules and decision tree. They can find the rules in the system and optimize the simulation system.We use association rule method to find rules in simulation experiment. We make some improvement in Apriori algorithm. First, use the vector data structure, reduce the space of the input dataset, so it can be put into the main memory and avoid the overhead caused by scanning databa