Spark高级数据分析-(影印版)

forewordpreface1. analyzing big data the challenges of data science introducing apache spark about this book2. introduction to data analysis with scala and spark scala for data scientists the spark programming model record linkage getting started: the spark shell and sparkcontext bringing data from the cluster to the client shipping code from the client to the cluster structuring data with tuples and case classes aggregations creating histograms summary statistics for continuous variables creating reusable code for computing summary statistics simple variable selection and scoring where to go from here3. recommending music and the audioscrobbler data set data set the alternating least squares recommender algorithm preparing the data building a first model spot checking recommendations evaluating recommendation quality computing auc hyperparameter selection making recommendations where to go from here4. predicting forest cover with decision trees fast forward to regression vectors and features training examples decision trees and forests covtype data set preparing the data a first decision tree decision tree hyperparameters tuning decision trees categorical features revisited random decision forests making predictions where to go from here5. anomaly detection in network traffic with k-means clustering anomaly detection k-means clustering network intrusion kdd cup 1999 data set a first take on clustering choosing k visualization in r feature normalization categorical variables using labels with entropy clustering in action where to go from here6. understanding wikipedia with latent semantic analysis the term-document matrix getting the data parsing and preparing the data lemmatization computing the tf-idfs singular value decomposition finding important concepts querying and scoring with the low-dimensional representation term-term relevance document-document relevance term-document relevance multiple-term queries where to go from here7. analyzing co-occurrence networks with graphx the medline citation index: a network analysis getting the data parsing xml documents with scala's xml library analyzing the mesh major topics and their co-occurrences constructing a co-occurrence network with graphx understanding the structure of networks connected components degree distribution filtering out noisy edges processing edgetriplets analyzing the filtered graph small-world networks cliques and clustering coefficients computing average path length with pregel where to go from here8. 6eospatial and temporal data analysis on the new york city taxi trip data getting the data working with temporal and geospatial data in spark temporal data with jodatime and nscalatime geospatial data with the esri geometry api and spray exploring the esri geometry api intro to geojson preparing the new york city taxi trip data handling invalid records at scale geospatial analysis sessionization in spark building sessions: secondary sorts in spark where to go from here 9. estimating financial risk through monte carlo simulation terminology methods for calculating var variance-covariance historical simulation monte carlo simulation our model getting the data preprocessing determining the factor weights sampling the multivariate normal distribution running the trials visualizing the distribution of returns evaluating our results where to go from here10. analyzing genomics data and the bdg project decoupling storage from modeling ingesting genomics data with the adam cli parquet format and columnar storage predicting transcription factor binding sites from encode data querying genotypes from the 1000 genomes project where to go from here11. analyzing neuroimaging data with pyspark and thunder overview of pyspark pyspark internals overview and installation of the thunder library loading data with thunder thunder core data types categorizing neuron types with thunder where to go from herea.deeper into sparkb.upcoming mllib pipelines apiindex