Text Mining Course

Lesson # 1: Introduction to Data & Text

Mining

Prof. Chiraz Ben Abdelkader

November 22, 2017

Outline

1) Introduction to data mining

2) Introduction to text mining

3) Overview of course

4) TP#1 (Intro to programming framework)

1) Intro. to Data Mining

What is Data Mining?

● The use of computational methods for the

extraction/discovery of novel, high-quality

information through automated processing of

large amounts of raw data.

● Examples of novel information: trends,

associations, predictive rules, etc.

● Heavily based on techniques from statistics and

machine learning

Why Data Mining?

● Because there is so much data! (data deluge)

● Data is very easily generated and stored

nowadays

● Data storage is so cheap (inexpensive)

● Infratsurcture for managing and processing

large amounts of data (cloud, data centers)

● Motto: store it now, use it later

Why Data Mining?

● Because Data != Knowledge

● We're drowning in data but starving for knowledge

● Raw data is useless, even if we have lots of it

Why Data Mining?

● Data ==> Knowledge

● Data contains hidden knowledge that could be

useful for solving problems and making decisions

● This knowledge is extracted by processing and

analyzing data in efficient & clever ways ...

DM = Opportunities and Challenges

● There is useful knowledge hidden inside these

massives amounts of data

● But this knowledge is difficult to find and extract

● Like mining for gold in a huge pile of useless sand

Examples of Data-driven Decisions

● Product sales: designing sales ("promotions");

product placement; stocking decisions; CRM

● Banks: deciding about approval of bank loans;

detecting credit card fraud

● Stock trade: deciding which stocks are most profitable

to buy/sell

● Meteorology and environmental science: predicting

weather; detecting changing weather patterns

● Genomics: finding genes that cause certain diseases

● Internet & Web: spam detection; Web search

engines; eCommerce; social media

Examples of Data-driven Decisions

● Product sales: designing sales ("promotions");

product placement; stocking decisions; CRM

● Banks: deciding about approval of bank loans;

detecting credit card fraud

● Stock trade: deciding which stocks are most profitable

to buy/sell

● Meteorology and environmental science: predicting

weather; detecting changing weather patterns

● Genomics: finding genes that cause certain diseases

● Internet & Web: spam detection; Web search

engines; eCommerce; social media

Homework Exercise

● For each example in the previous slide:

● What is the raw data?

● What is the hidden information (knowedge)?

● Data = what you directly measure or observe

● Information = what you want to know based on this

data

Fundamental Elements of DM

● Massive amounts of data

● Why massive? Because small data is inadequate to

find useful knowledge

● Automated processing of data

● Why automated? Because data is too big to

process manually

● The discovered patterns are non-trivial and

previously unknown

● Otherwise we would just call it simple data analysis

Patterns

● DM is sometimes called Pattern Discovery

● Knowledge extracted from data is formally

called patterns

● Pattern = a mathematical model of relationships

or associations between entities in the data

● Intuitively, a pattern is a concise way of making

useful generalizations based on data

● Useful analogy: a child learns to speak from

examples (i.e. by observing how others speak)

Types of Patterns

● Patterns can be broadly classified into 2 types

according to how they are used in an application

(Description)

Descriptive vs. Predictive Patterns

● Descriptive: models

used to describe

relationships in data

and draw conclusions.

● Predictive: models

used to estimate (or

predict) unknown

properties.

● Two major types:

1) Clustering

2) Association rules

1) Classification

2) Regression

Predictive Modeling

● The study of predictive patterns in general

● Mainly based on techniques from supervised

learning, a sub-discipline of machine learning

● Two main types of predictive patterns:

1) Classification: when the predicted property is

categorical, e.g. a person's gender

2) Regression: when the predicted property is

numerical, e.g. a person's height

Example Applications

Application

Pattern type

Input data

Customer segmentation

Clustering

Customer records

Spam detection

Classification

Emails

Handwriting recognition

Classification

images of handwritten text

Medical diagnosis

Classification

symptoms

Sentiment analysis

Classification

Movie reviews; social media comments; …

Face recognition

Face detection

Prediction of stock prices

Prediction of house prices

Prediction of temperature

(meteorology)

Example Applications

Application

Pattern type

Input data

Customer segmentation

Clustering

Customer records

Spam detection

Classification

Emails

Handwriting recognition

images of handwritten text

Medical diagnosis

Sentiment analysis

Face recognition

Face detection

symptoms

Classification

Each one of these

Classification

applications involves first

extracting a pattern (model)

based on observed data

(training data), then using

this model to drive the

application.

Prediction of stock prices

Prediction of house prices

Prediction of temperature

(meteorology)

Movie reviews; social media comments; …

Spam Detection

Handwriting Recognition

● Automatically convert image of handwritten text

into electronic format (as a text file)

Sentiment Analysis

● "The process of automatically identifying and

categorizing opinions expressed in a piece of

text, in order to determine whether the writer's

attitude towards a particular topic, product, etc.

is positive, negative, or neutral." [Wikipedia]

● Commonly used by companies to monitor their

reputation and reputation of their products

based on social media data

● e.g., Ooredoo & Orange recently started doing this

Example: movie reviews

Input: a piece of text

e.g. a tweet or movie

review

Output:

user’s

attitude

https://www.slideshare.net/PradeepPujari/sais-20431863

Medical Diagnosis

Face Recognition

https://leaksource.files.wordpress.com/

Input

Image containing

human face(s)

Output

ID or name of

person(s) in image

Face Detection

http://cs.brown.edu/courses/cs143/2011/results/proj4/hangsu/

Input

Image

Output

Location of human

faces in image

Example Applications

Application

Pattern type

Input data

Customer segmentation

Clustering

Customer records

Spam detection

Classification

Emails

Movie reviews; social media comments; …

Handwriting recognition

images of handwritten text

symptoms

Classification

Each one of these

Classification

applications involves first

extracting a pattern (i.e.

model) based on observed

data (training data), then

using this model to drive

the application.

Medical diagnosis

Sentiment analysis

Face recognition

Face detection

Prediction of stock prices

Prediction of house prices

Prediction of temperature

(meteorology)

Extracting vs. Using Patterns

Extract pattern from data (build or fit model to data)

Pattern

Extraction

Algorithm

Model

Training Data

Use or interpret the extracted pattern

Using a

predictive

pattern

Test Data

Model

Predicted /

Estimated

Data Values

Example: Face Recognition

● Extracting the pattern:

● What kind of pattern do we need for this problem?

● What is the training data?

● Using the pattern:

● How is it used?

● In what kind of application?

2) Intro. to Text Mining

What is Text Mining?

● Text mining = data mining for text data

● So TM is a sub-field of Data Mining

● Text data = anything written by humans for

other humans to read; i.e. written in a human

language (natural language), such as Arabic

Why is TM Important?

● Text data is everywhere nowadays; available in

huge and ever increasing quantities

● Effective and efficient exploitation of such huge

quantities of textual content

● Need computational tools to automate this

process

Examples of Text Data

● Books

● Scientific articles

● news articles

● Email

● SMS (mobile phone

messages)

● Web pages

● blog posts

● Social media messages /

comments / status

updates

● financial and business

reports

● administrative documents

● Insurance claims

● Patent portfolios

● Customer complaint letters

● Contracts

● Transcripts of phone calls

with customers

● ...

Example Applications of TM

● Spam detection in gmail and yahoo mail

● Analyzing web sites of competitors to compare prices

● Translation of Web pages in Google chrome and of

user comments on Facebook

● Investigate public opinion on social media about a

particular product, brand, company, or political party

● Recommend related articles, groups on Facebook

● Analyzing patient records to detect abnormalities

● Analyzing scientific articles related to particular topic to

find latest findings

Relationship between TM & NLP

● TM: automated processing of text data to

extract patterns (knowledge) from text

● NLP (natural language processing): automated

processing of human language (in any form:

text, audio, images) in order to understand its

content, i.e. extract its meaning

● TM uses NLP techniques, obviously because

extracting knowledge requires some level of

understanding of the meaning of the text

Some Major NLP Tasks

● Speech synthesis from text (text to audio)

● Speech recognition (audio to text)

● Translation

● Language identification

● Sentiment analysis

● Text summarization

● Text categorization (grouping documents into

separate categories)

Challenges of NLP & TM

● Human language text is inherently difficult for a

computer to understand

● That is, the task of writing computer programs that

can determine meaning of such text is challenging

● This is mainly because of:

1) Ambiguities in text (fuzzy meaning)

2) Errors (typos and linguistic mistakes)

3) Use of informal language (e.g. Arabizi, slang,

abbreviations in social media comments )

● These challenges are usually handled by

exploiting linguistic knowledge and context

Human language is ambiguous

Police squad help dog bite victim

Source: https://www.slideshare.net/falah_hasan77/some-problems-of-ambiguity-in-translation-with-reference-to-english-and-arabic

Human language is ambiguous

● at the word level e.g. "run"

● at the sentence level

● Punctuation

● (Lack of) "tachkil" in Arabic

● sarcasm

● Idioms

● context

Python

Avocado

I saw the man with the telescope

The chicken is ready to eat

He is calculating

Old men and women

Time flies like an arrow

Fruit flies like bananas

This was a great movie, yea right!

Challenges (cont.)

Source: Prof. Dan Jurafsky, Stanford University

3) Overview of this Course

Course Schedule

1. Introduction to DM & TM + TP (Python & Jupyter)

2. Document representation + TP

3. Document classification + TP

4. Topic modeling + TP

5. Project presentations & test

● First 1.5 hours: lesson

● Second 1.5 hours: TP using Python & Jupyter

Grade Distribution

● 45% final programming project

● 35% written test (1 hour)

● 20% attendance & TP

● absence and late arrival will both be penalized

● Programming exercises in Python & Jupyter

● Each TP must be finished during class

● You will need your Personal Computer for TP

● You can work in pairs during TP (one PC per

pair)

Final Programming Project

● Group project (2-4 students per group)

● Must be implemented in Python

● Project proposal is due on week#4 (Dec. 15)

● Project report & validation are due on

week#7 (Jan. 05)

4) TP1

Programming Framework

● Python (Version 3)

● Jupyter IDE

Before TP#2 (next session), make sure you :

1. have Python and Jupyter installed on your PC.

2. know how to create and edit Jupyter notebooks.

3. are comfortable writing and editing basic Python programs in Jupyter.

Why Python?

● Easy to learn

● Widely used for data science in general and for

text mining in particular

● Tons of free libraries

Source: https://dzone.com/articles/which-are-the-popular-languages-for-data-science

What is Jupyter?

● One of the popular IDEs for Python

● Open-source

● Well suited for Python beginners, collaboration,

rapid prototyping

● But not for heavy code development &

production

Source: https://www.oreilly.com/ideas/what-is-jupyter

What is Jupyter?

"Built for writing and sharing code and text,

within the context of a web page [the client].

The code runs on a server, and the results are

turned into HTML and incorporated into the

page you're writing. That server can be

anywhere: on your laptop, behind your firewall,

or on the public internet. Your page contains

your thoughts [explanations], your code, and

the results of running the code."

Source: https://www.oreilly.com/ideas/what-is-jupyter

Jupyter Notebooks

● A Jupyter notebook is a .ipynb file

● Contains Python code, Python output, and free

text (explanations, comments)

● Can be viewed & edited with any Web browser

Example Notebooks

TP # 1

1) Install Anaconda + Python (Version 3) + Jupyter

http://jupyter.readthedocs.io/en/latest/install.html

https://www.youtube.com/watch?v=HW29067qVWk

https://www.youtube.com/watch?v=Q0jGAZAdZqM

2) Start Jupyter program

3) Rename your first notebook as TP1

Change this to TP1

TP#1 (cont.)

3) Do one of the following Python tutorials.

● Official Python tutorial (Sections 1 – 7)

https://docs.python.org/3/tutorial/index.html

● Data camp tutorial ("Learn the Basics" Section)

https://www.learnpython.org/

Before TP#2 (next session), make sure you :

1. have Python and Jupyter installed on your PC.

2. know how to create and edit Jupyter notebooks.

3. are comfortable writing and editing basic Python programs in Jupyter.

Useful Links for Jupyter

● Jupyter Tutorial

https://www.datacamp.com/community/tutorials/tutorial-jupyter-

notebook

● Running Jupyter

http://jupyter.readthedocs.io/en/latest/running.html

● Jupyter "Cheat sheet" (quick reference to major

commands)

https://www.datacamp.com/community/blog/jupyter-notebook-ch

eat-sheet

● Jupyter Official Website

http://jupyter.org/

Useful Links for Python

● Popular online tutorials for learning Python

https://www.datacamp.com/courses/intro-to-python-for-d

ata-science/

https://www.learnpython.org/

https://www.tutorialspoint.com/python/

https://learnpythonthehardway.org/book/ex1.html

https://developers.google.com/edu/python/

The End