Natural Language Inference (NLI) is a task in natural language processing that involves determining the logical relationship between two given text segments. Typically, these segments are a premise and a hypothesis. The goal is to classify the relationship as either “entailment,” “contradiction,” or “neutral.” NLI is often used to evaluate the understanding of language semantics and reasoning capabilities of language models. Models pretrained on large language datasets, such as BERT, have demonstrated strong performance on NLI tasks by capturing contextual information and relationships between sentences.

Posts

Overcoming Poor Word Embeddings with Word Definitions

/in /by

Modern natural language understanding models depend on pretrained subword embeddings, but applications may need to reason about words that were never or rarely seen during pretraining. We show that examples that depend critically on a rarer word are more challenging for natural language inference models. Then we explore how a model could learn to use definitions, provided in natural text, to overcome this handicap. Our model’s understanding of a definition is usually weaker than a well-modeled word embedding, but it recovers most of the performance gap from using a completely untrained word.

Read more

Teaching Syntax by Adversarial Distraction

/in /by

Existing entailment datasets mainly pose problems which can be answered without attention to grammar or word order. Learning syntax requires comparing examples where different grammar and word order change the desired classification. We introduce several datasets based on synthetic transformations of natural entailment examples in SNLI or FEVER, to teach aspects of grammar and word order. We show that without retraining, popular entailment models are unaware that these syntactic differences change meaning. With retraining, some but not all popular entailment models can learn to compare the syntax properly.

Read more