MIT-Machine-learning-for-nlp-Michael Collinsl8
- 格式：pdf
- 大小：171.36 KB
- 文档页数：60
¯ Step 1: represent a tree as a sequence of decisions
Ì
½ ¾
Ñ
½
Ñ
¯ First Ò decisions are tagging decisions
½
Ò
DT, NN, Vt, DT, NN, IN, DT, NN
Layer 2: Chunks
NP DT the NN lawyer Vt questioned DT the NP NN witness IN about DT the NP NN revolver
NP NN revolver
Check=NO
Start(S) NP DT the NN lawyer
Start(VP) Vt questioned
Join(VP) NP DT the NN witness
Start(PP) IN about DT the
NP NN revolver
Start(S) NP DT the NN lawyer
A General Approach: (Conditional) History-Based Models
¯ Step 1: represent a tree as a sequence of decisions
Ñ is not necessarily the length of the sentence Ì
NP DT the NN witness IN
PP NP NN revolver
about DT the
S
NP DT the NN lawyer Vt questioned DT the
VP
NP NN witness IN
PP NP NN revolver
about DT the
Check=YES
Join(VP) NP DT the NN witness
Start(PP) IN about
Join(PP) NP DT the NN revolver
Start(S) NP DT the NN lawyer
Start(VP) Vt questioned
Join(VP) NP DT the NN witness IN
A General Approach: (Conditional) History-Based Models
¯ Step 1: represent a tree as a sequence of decisions
Ñ is not necessarily the length of the sentence Ì
½ ¾
Ñ
½
Ñ
¯ Step 2: the probability of a tree is Ñ È ´Ì Ë µ È´
½
½
½ Ë µ
¯ Step 3: Use a log-linear model to estimate È´ ½ ½ Ë µ ¯ Step 4: Search?? (answer we’ll get to later: beam or heuristic
½ ¾
Ñ
½
Ñ
¯ Step 2: the probability of a tree is Ñ È ´Ì Ë µ È´
½
½
½ Ë µ
¯ Step 3: Use a log-linear model to estimate È´ ½ ½ Ë µ ¯ Step 4: Search?? (answer we’ll get to later: beam or heuristic
Start(S) NP DT the NN lawyer Vt questioned DT the
VP
NP NN witness IN
PP NP NN revolver
aboutБайду номын сангаасDT the
Check=YES
Start(S) NP DT the NN lawyer Vt questioned
Join(S) VP
IN about DT the
NP NN revolver
Start(S) NP DT the NN lawyer
Vt questioned DT the
NP NN witness
IN about DT the
NP NN revolver
Start(S) NP DT the NN lawyer
Vt questioned DT the
Log-Linear Taggers: Independence Assumptions
¯ The input sentence Ë , with length Ò
possible tag sequences.
Ë Ð Ò Ø , has
Ì
Ò
¯ Each tag sequence Ì has a conditional probability ÉÒ È ´Ì Ë µ ½ È ´Ì ´ µ Ë Ì ´½µ Ì ´ ½µµ Chain rule
Start(VP) Vt questioned
Join(VP) NP DT the NN witness
Start(PP) IN about DT the
NP NN revolver
Check=NO
Start(S) NP DT the NN lawyer
Start(VP) Vt questioned
Meaning of these actions:
¯ Start(X) starts a new constituent with label X (always acts on leftmost constituent with no start or join label above it) ¯ Join(X) continues a constituent with label X ¯ Check=NO does nothing ¯ Check=YES takes previous Join or Start action, and converts
6.891: Lecture 8 (October 1st, 2003) Log-Linear Models for Parsing, and the EM Algorithm Part I
Overview
¯ Ratnaparkhi’s Maximum-Entropy Parser ¯ The EM Algorithm Part I
Ratnaparkhi’s Parser: Three Layers of Structure
1. Part-of-speech tags 2. Chunks 3. Remaining structure
Layer 1: Part-of-Speech Tags
DT the NN lawyer Vt questioned DT the NN witness IN about DT the NN revolver
Ò
A General Approach: (Conditional) History-Based Models
¯ We’ve
shown how to define sequence
È ´Ì
Ë µ where Ì is a tag
¯ How do we define È ´Ì
structure)?
Ë µ if Ì is a parse tree (or another
NP NN witness
IN about DT the
NP NN revolver
Check=NO
Start(S) NP DT the NN lawyer
Start(VP) Vt questioned DT the
NP NN witness
IN about DT the
NP NN revolver
¯ Step 1: represent a tree as a sequence of decisions
Ì
½ ¾
Ò
½
Ò
¯ First Ò decisions are tagging decisions
½ ¾Ò
Next Ò decisions are chunk tagging decisions DT, NN, Vt, DT, NN, IN, DT, NN, Start(NP), Join(NP), Other, Start(NP), Join(NP), Other, Start(NP), Join(NP)
it into a completed constituent
(always acts on leftmost constituent with no start or join label above it)
NP DT the NN lawyer
Vt questioned DT the
NP NN witness
PP NP NN revolver
about DT the
Check=YES
Start(S) NP DT the NN lawyer
Start(VP) Vt questioned
Join(VP) NP DT the NN witness
Join(VP) PP IN about DT the NP NN revolver
Chunks are defined as any phrase where all children are partof-speech tags (Other common chunks are ADJP, QP)
Layer 2: Chunks
Start(NP) DT the Join(NP) NN lawyer Other Vt questioned Start(NP) DT the Join(NP) NN witness Other IN about Start(NP) DT the Join(NP) NN revolver
