Loading [MathJax]/extensions/TeX/AMSsymbols.js
Skip to main content

NLG pipeline

NLGΒΆ

NLG stands for Natural Languate Generation. NLG is one field of AI aims to generate the understandable and appropriate texts from raw data. We should differientiate the concepts of NLG with NLP and NLU. NLP is natural languate processing. This is a field in AI working on text generally. NLP contains Speed recornigion, Speed synthesis, NLG and NLU. NLU and NLG is subsets of NLP. While NLG generate the text, NLU uses text as input and generate some pattern such as Sentiment Analysis, Summary.

The pipeline of NLG

NLG can be divided into 3 phases: Document planning, Microplanning and Realisation. The purpose of Document planning is to chose what to say and the purpos of Microplanning and Realisation is to find how to say. There are some components in each phase. In traditional NLG system, we have 5 components: Content Determination, Text Structure, Aggregation, Lexicalisation, Reffering expression, Realisation.

Content Determination

Content Determination is sets of entities, concepts, relation that we want to talk about in our content. Content determination is the most important part in every NLG engine. It defines the ideas of text and the purpose of context. The input of Contet determination is preporocessed data. NLG engine needs domain knowlede of expert to detect the appropriate information. eg: in the financial commentary, we have to detect the most loss Product. It will determine the reason of P&L loss. Besides the domain knowledge, we also use the experience or use corpus to detect content. Some NLG engine also builds ontology and based on this metatdata to detect content.
There are some approaches for content determination. Basically, we can use rule-based for this one. It is based on some constraint and condition to detect content from preprocessed data. Another approach is using Machine Learning. Some techniques in Machine learning are clustering or classification the data.

Text Structure

Text Structure aims to order the messages from content determination component. It helps to build structure for our generated text. The structure should be consider with these aspects:
  • Group: grouping the text
  • Order: the flow or the text
  • Relation: relation between element in text
There are 2 main approaches for text structure:
  • using Schema
  • Using Rhetorial Structure Theory

Aggregation

Aggregation receives input from Text Structure and refine the input in concrete way. There are some strategy to aggregate the text structure:
  • by conjunction
  • by share participant
  • by shared object
  • embeded properties
NLG egine also uses some approaches for aggregation:
  • by rule-based
  • by Machine Learning
  • by Graph

Lexicalisation

Lexicalisation aims to generate words and syntatic for our generated text. NLG engine considers to Lexicalisation in 2 aspects:
  • Coarse grained lexicalisation
  • Fined grained lexicalisation
The most popular approach for lexicalisaion is based on graph. NLG engine will build a graph to determine word.

Referring expression

Referring expression aims to add more informaiton of entity, concept or relation into generated text. Some approaches of referring expression:
  • init introduction
  • pronoun
  • definite description (explain)

Realisation

Realisation is the final step of NLG pipeline. It helps to genete the text and show meaningful info for user. Realisation mentions about:
  • verb formution
  • structure aggrement
  • syntatically
There are some approach for this component:
  • inversion parser
  • Systemic grammar
  • Meaning-text grammar
  • Template
End To End methodology
There are some researches which try to buid the NLG system in only one component with deeplearning. It means the system does not have several components and long pipeline anymore. what it needs is the input and it will produce the final NLG text or output. It's a reall interesting approach and it has some results in reaserch only. However, there are some limited of this approach:
- the generated text might be not totally correct.
- generating unnecessary text 
- the model needs a lots of data.

Comments

Popular posts from this blog

Generative Adversarial Networks

Generative Adversarial Networks (GAN) is one a Neural Network architecture which simulates zero-sum game. There are 2 parts of this Neural Network. The first is called Generator and other is called Discriminator. Generator tries to mimic data and make the fake data likes the real data in distribution. Meanwhile, the Discriminator tries to maximize the difference between real data and fake data. It is reason we call zero-sum game. Two parts are coaction with each other. This structure makes the GAN to be a interesting Neural Network architecture and it has many application in both academic and industry. In modeling, the GAN is an approach of equilibrium Networks such as Boltzmann Machine did. It is an optimization problem with objectives of: minimize Generator and maximum Discriminator simultaneously.  $max_{D}min_{G} V(D,G)$  $max_{D}min_{G} V(D,G) = E_{x\sim p_{data}(x)}[log(D(x))] + E_{z\sim p_{z}(z)}[log(1 - D(G(z)))]$ $V(D,G)$ is optimization problem subject to G ...

Mutual information and feature selection

   Feature selection is one of the most important step to make your model works well. In data mining, feature selection is the first step and it effects to all of process. Feature selection help model on some points: - The model will be trained faster - Reduce overfitting - Simplifying model - Reduce the dimension of data Hence, feature selection is kick-off step and it effects overall, especially in model. There are 3 type of feature selection: Filter methods, wrapper methods and embedded methods. Filter methods: this methods "filter" data based on correlation score. Normally, our data have many features, and a label. We calculate the correlations between features and label. After that, we only retrain the features that have a good (relevant) correlated score and remove others. In this type of method we have some ways to calculate the correlation. - Pearson correlation: this one is based on covariance between 2 continuous variables. $$ p_{X,Y} = \frac {Cov(X, Y)...

The basic concepts of deep neural networks

We will explore 3 basic concepts of deep neural networks. The first part will explain how back-propagation algorithm works. The second talks about softmax function. And the last one is cost function. Inside Backpropagation algorithm The main idea of Backpropagation algorithm is aims to minimize the overall neural network error. To do that, we find out the partial derivative of error and weight vector. Take a look in a simple Neural Network: From Sigmoid function to Softmax function and Cross-entropy cost function In the classification problem with multinomial distribution, how do we know which label is choosen? Softmax function is used to do that. In practice, some people might also use one2All mechanism to detect label by traning many logistic regression models. However, it is not effective solution. Softmax function helps us to determine the label also we can calculate the cost from this easily by using Cross-entropy cost function. Sigmoid function Sigmoid function co...