I am a PhD student in the Computer Science department at the University of Oxford, advised by Prof. Varun Kanade and Prof. Phil Blunsom. My PhD is generously supported by Google DeepMind. I am broadly interested in the intersection of theoretical and empirical/scientific understanding of deep learning models. Most of my research focuses on analyzing the expressiveness and algorithmic learning abilities of neural network architectures to gain insights that can help us develop more effective models.
Keywords: Expressivity, Science of Deep Learning, Algorithmic Reasoning, Transformers, RNNs/SSMs
I am currently a student researcher at Google in Sunnyvale, where I am working on improving LLM agents. Over the last two summers, I interned at Cohere, where I worked on pretraining LLMs with non-Transformer architectures. Before joining Oxford, I spent two amazing years as a Research Fellow at Microsoft Research India, where I worked with Dr. Navin Goyal. Prior to that, I spent a wonderful semester working with Dr. Partha Talukdar at the Indian Institute of Science. I graduated with a B.E. (Hons.) in Computer Science and an Int. M.Sc. (Hons.) in Biological Science from BITS Pilani, India, in 2019. For more details, refer to my CV or drop me an email.
Separations in the Representational Capabilities of Transformers and Recurrent Architectures
, Michael Hahn, Phil Blunsom, Varun Kanade
NeurIPS 2024
pdf
abstract
Understanding In-Context Learning in Transformers and LLMs by Learning to Learn Discrete Functions
, Arkil Patel, Phil Blunsom, Varun Kanade
ICLR 2024 Oral
pdf
code
abstract
On the Ability and Limitations of Transformers to Recognize Formal Languages
, Kabir Ahuja, Navin Goyal
EMNLP 2020
pdf
code
abstract
On the Practical Ability of RNNs to Recognize Hierarchical Languages
Best Short Paper Award
, Kabir Ahuja, Navin Goyal
COLING 2020
pdf
code
abstract
LibNMF
An easy to use python library with implementations of a set of tested optimization and regularization methods of NMF. Implemented Algorithms include graph regularized NMF, probabilistic NMF, a first-order primal-dual algorithm ...etc
Github
PyDPP
A python package available in pip with modules for sampling from Determinantal Point Processes (DPP). Contains implementations of algorithms to sample from DPPs that encourage diversity in the selection of a subset of points from a grounded superset.
Github
- Fall 2023: Departmental Tutor for Computational Learning Theory , University of Oxford
- Fall 2022: Departmental Tutor for Computational Learning Theory , University of Oxford
- Spring 2018: Teaching assistant for Neural Networks and Fuzzy Logic , BITS Pilani
