Universal Transformers

Thanks to Stephan Gouws for his help on writing and improving this blog post. Transformers have recently become a competitive alternative to RNNs for a range of sequence modeling tasks. They address a significant shortcoming of RNNs, i.e. their inherently sequential computation which prevents parallelization across elements of the input sequence, whilst still addressing the […]

Fidelity-Weighted Learning

Our paper “Fidelity-Weighted Learning”, with Arash Mehrjou, Stephan Gouws, Jaap Kamps, Bernhard Schölkopf, has been accepted at Sixth International Conference on Learning Representations (ICLR2018). \o/ [perfectpullquote align=”full” bordertop=”false” cite=”” link=”” color=”” class=”#16989D” size=”16″] tl;dr Fidelity-weighted learning (FWL) is a semi-supervised student-teacher approach for training deep neural networks using weakly-labeled data. It modulates the parameter updates to […]

Learning to Attend, Copy, and Generate for Session-Based Query Suggestion

Our paper “Learning to Attend, Copy, and Generate for Session-Based Query Suggestion”, with Sascha Rothe, Enrique Alfonseca, and Pascal Fleury, has been accepted as a long paper at the international Conference on Information and Knowledge Management (CIKM’17). This paper is on the outcome of my internship at Google Research. \o/ Users interact with search engines […]

Share your Model instead of your Data!

Our paper “Share your Model instead of your Data: Privacy Preserving Mimic Learning for Ranking”, with Hosein Azarbonyad, Jaap Kamps, and Maarten de Rijke, has been accepted at Neu-IR: SIGIR Workshop on Neural Information Retrieval (NeuIR’17). \o/ [perfectpullquote align=”full” cite=”” link=”” color=”” class=”” size=”14″]In this paper, we aim to lay the groundwork for the idea […]

Beating the Teacher: Neural Ranking Models with Weak Supervision

Our paper “Neural Ranking Models with Weak Supervision”, with Hamed Zamani, Aliaksei Severyn, Jaap Kamps, and W. Bruce Croft, has been accepted as a long paper at The 40th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR2017). \o/ This paper is on the outcome of my pet project during my internship […]