Category: NLP

Machine Learning, NLP

Speed benchmark einsum vs matmul in XL-Attention

The original Transformer can only attend to a fixed and limited segment of the input to compute its attention. The major drawback of this architecture is that no information can flow across separated segments which prevents the Transformer to model long-term dependencies. Transformer-XL is an enhancement to the vanilla Transformer which enables the latter to store the most recent hidden states in a fixed-length memory … Continue reading Speed benchmark einsum vs matmul in XL-Attention

Davide LiuLeave a comment
Machine Learning, NLP

SeqGAN: text generation with generative models

In this post, we propose to review the recent history of research in the Natural Language Generation (NLG) tasks of the Natural Language Processing domain. Realistic human-like language generation has been a challenge for researchers that has recently come into greater focus with the release of large neural models for NLP like the GPT and BERT models. In this post, we propose to focus on … Continue reading SeqGAN: text generation with generative models

Davide LiuLeave a comment
Machine Learning, NLP

Sentences sentiment analysis with CNN

Opinion mining (sometimes known as sentiment analysis or emotion AI) refers to the use of natural language processing, text analysis, computational linguistics, and biometrics to systematically identify, extract, quantify, and study affective states and subjective information. Sentiment analysis is widely applied to the voice of the customer materials such as reviews and survey responses, online and social media, and healthcare materials for applications that range … Continue reading Sentences sentiment analysis with CNN

Davide Liu1 Comment
Machine Learning, NLP

Word similarity and analogy with Skip-Gram

In this post, we are going to show word similarities and word analogies learned by 3 Skip-Gram models trained to learn word embedding from a 3GB corpus size taken scraping text from Wikipedia pages. Skip-Gram is unsupervised learning used to find the context words of given a target word. During its training process, Skip-Gram will learn a powerful vector representation for all of its vocabulary … Continue reading Word similarity and analogy with Skip-Gram

Davide LiuLeave a comment