A Transformer Network refers to a type of neural network architecture in which transformers have become a foundational architecture in natural language processing (NLP) and have been applied to various other domains, including computer vision. Transformer architecture has become a cornerstone in deep learning and has significantly impacted the field, leading to improvements in various natural language processing and computer vision applications. Its success is attributed to its ability to capture long-range dependencies, parallelize computation, and facilitate efficient training on large datasets.

Posts

Teaching Syntax by Adversarial Distraction

/in /by

Existing entailment datasets mainly pose problems which can be answered without attention to grammar or word order. Learning syntax requires comparing examples where different grammar and word order change the desired classification. We introduce several datasets based on synthetic transformations of natural entailment examples in SNLI or FEVER, to teach aspects of grammar and word order. We show that without retraining, popular entailment models are unaware that these syntactic differences change meaning. With retraining, some but not all popular entailment models can learn to compare the syntax properly.

Read more