AWS动态管理大规模Spark集群.pptx

AWS动态管理大规模Spark集群 技术创新 变革未来 Founded in 2013 with HQ in Mountain View, California USA Strong financial backing over $50M by Sequoia Capital, Genesis Capital, and GSR ventures Technology 7 patent pending Elite tech-team of PhDs from top universities specializing in machine learning, big data and security Clients Partnered with global clients with an online or mobile presence in gaming, social, commerce and finance 4B+ Protected accounts globally 800B+ Processed events to date 200M Detected + bad accounts to date Rules Engines Supervised Machine Learning Unsupervised Machine Learning Reputation Lists How it works Search reputation database Matches against lists Make decision Sende r Receiver Reputation Database Is Sender IP Listed? YES / NO Make Decision Examples Email IP Address Devices Credit Card #’s Phone #’s How it works Check against rule lists Criteria with weights Combination rules with logic IF (user email = free email service) AND (comment character count ≥ 150 per sec) { flag user account as spammer mute commenting } RULE WEIGHT IP Address is anonymous proxy +800 Account age 180 days -500 Email is private corporate domain name -350 Mismatch billing country and IP country +450 Phone number found on 3 accounts +250 What is it? An algorithm that learns to perform a task from known examples (training data). An important requirement of using supervised learning is having the data to train the model. What is it? An algorithm that learns to identify linkages and patterns in the data without prior knowledge of what to look for. Unsupervised machine learning does not require labeled training data. Comparison Between Approaches Time Reputation List and Device Fingerprint Rules Engine Supervised Machine Learning Unsupervised Machine Learning Effectiveness ［－］ Limited coverage and precision ［－］ Can use emulators to bypass device fingerprint ［－］Need to maintain and adapt rules constantly ［－］Poor against adaptive attacks ［－］ Need large amount