Lecture Two: Content Analysis and Natural Language Processing I

Lecture Summary

1. Content analysis
1. Analytical technique that can be used to extract secondary meaning from information
2. Vocabulary Analysis - examine documents purely as a collection of words and look at patterns
3. Readability Indexes - the ease with which a native speaker of average intelligence can read and understand a passage of text
4. Normative Comparison - compare documents to some standard of normal text
2. Natural Language Processing
1. Make the computer understand natural language input and generate natural language output
2. How is NLP done?
• Pre-Processing - remove useless parts of the documents
• Tokenization - break the raw text into meaningful units
• Normalization - e.g. cases
• Stemming - reduce words of different forms to the "root"
• Parts of Speech (POS) Tags
• Language Model - probabilitic models of sequences of words; enables word predictions
• unigram, bigram, trigram...
• p(wi) = c(wi)/N
• p(wi|wi-1) = c(wi-1wi)/C(wi-1)
• p(wi|wi-2wi-1) = c(wi-2wi-2wi)/C(wi-2wi-1)
3. Google Book’s N-Gram Viewer

Notes

The main dish of this lecture is the steps of doing NLP. As a developer, most of the works can be done by simply calling libraries. However, we do need to understand the theories behind them. Among the introduced steps, I paid more attention on the N-Gram models. It is interesting to have computer predicting something that it doesn't really understand. I hope I can have chance to use those models in real life cases.
N-Gram Viewer is an interesting source of research. I will keep an eye on it.

Lecture One: Fundamentals of Social Media and Social Media Analytics

In the first lecture, the fundamentals of Social Media and Social Media Analytic were introduced.

Lecture Summary

 This lecture included the following areas:

1. What are social networking and social media?
2. The key technology that empowers social media - Web 2.0.
3. The study of social media - science and engineering
4. Understanding the Users
1. Behaviorist Approach
2. Social Cognition Approach
5. Dokdo/Takeshima - An example of wikipedia
6. Social media analytic - concerned with developing and evaluating informatics tools and frameworks to extract, analyze, summarize, and visualize information
1. Not only data mining and statistical analysis
2. An interdisciplinary area involving BOTH humanities and technologies
7. Social media analysis - focus on the “analysis” component within social media analytics
1. Aims for conducting social media analysis
8. Work flow of social media analytic
1. Social media
2. Data collection
3. Analysis and development
4. Interpretation and presentation of results
5. Applications / Decision-making / Prediction / Research Findings
9. Some case studies

Notes

This lecture was essentially a high level overview of whole course.

Of the whole lecture, to me, the most difficult idea is social cognition. It is a new term that I have never met before so I spent some time to understand its meaning.

Social Cognitive Theory (SCT) - People continue to influence and be influenced by the social environment surrounding them

 I wonder if I misunderstood this theory. It looks like common sense. I cannot think of how to apply this theory in the analysis. 

My First Blog Post

Hello everyone! Welcome to my first blog!

Blog is an excellent place for consolidating newly learnt knowledge. Not only can the writer have a chance to revise the new knowledge, the others can also exchange ideas with the writers easily. It is beneficial to both the writer and the readers.

I would like to use this place to summarize what I have learnt in the Social Media Analytic course.