Initial run
(Suzy's system except for gathering process)
| Query | Good | Bad | Both wrong | Bad sentence | Not sure |
| Alexander Graham Bell invented telephone | 7 | 5 | 3 | 2 | 0 |
Comments
paused to collect results properly
Problems
- Alexander Graham Bell was a Scot who invented the telephone . (i.e. rel clause, tries to hang it off Bell rather than Scot)
- need to resolve anaphora? Or excluse who/whose/he/him/her/his sentences
- remove ':' sentences and '?'s? (But ':' seem also to involve many improvements...)
- error applying constraint to 'In short , it will be the greatest thing since Alexander Graham Bell invented the telephone in the 19th century .' (sim to Edmund Barton problem)
Experiment 1
Question: does it improve general performance?
Gen method:
- automatically extract queries from freebase for (composer, bday), (inventor, invention), (director, film)
- get 'top 40' (based on yahoo hit counts), and manually winnow 'bad' ones
- extract constraints that are shared across queries, only from sentences < 15 tokens and 1 to 2 verbs (have to have two verbs, otherwise miss some nice ones).
- apply nice constraints (e.g. containing a word >= 3 chars to avoid punc) that have 'maximal' match (e.g. if we have a word match on 4 constraints, ignore 3 constraint options) to all sentences; general our 'nice' output
Eval:
- Train models on CCG bank subset +
- nothing
- constrained sentences
- unconstrained sentences (i.e. a lot more data than constrained sentences)
- Eval on Steve's wikipedia sentences + standard WSJ section
Results: ?
