Context

———————————–

Data munging.

obtain, scrub, explore, model and interpret data, blending hacking, statistics and machine learning.

Data wrangling/data jujitsu/data munging.

three types of tasks:

1. preparing to run a model.(gathering, cleaning, integrating, restructuring, transforming, loading, filtering, deleting, combining, merging, verifying, extracting, shaping, massaging)

2. running the model

3. communicating the results

Data Products

Data-driven apps.(spellchecker, machine translator)

interactive visualization. (google flu application, global burden of disease)

online databases. (enterprise data warehouse, sloan digital sky survey)

—————————–

$Dimensions$

Distinguish Data Science from BI, statistics, data management, visualization, and machine learning.

Screen Shot 2014-07-20 at 11.24.30 PM

 

What do you need to do:

1. learn to deal with unstructured data,

2. data not fit in memory

3. statistical modeling and how to communicate results

4. algorithms and trade offs at scale

Breadth

tools <————-> abstraction(Implementation)

Hadoop               MapReduce

PostgreSQL        Relational Algebra

glm(..) in R           Logistic Regression

Tableau               InfoVis

Depth

Structures <———-> statistics

Management             Analysis

Relational Algebra     Linear Algebra

Standards                  ad hoc files

Scale

desktop <————-> cloud

main memory           distributed

R                              Hadoop

local files                  S3, Azure storage

Target

hackers <————-> analysts

assume                      assume little programming

proficiency in

python, java, R

—————————————

eScience = Data Science

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s