Publications
List of my research publications and preprints
MultiNet v1.0: A Comprehensive Benchmark for Evaluating Multimodal Reasoning and Action Models Across Diverse Domains
Our findings reveal significant insights into the current state of multimodal AI, highlighting both promising capabilities and critical limitations that inform future research directions. We release our complete benchmark suite, evaluation framework, and detailed analysis to accelerate progress in this field.
Confirmation bias: A challenge for scalable oversight
We conducted two studies examining the performance of simple oversight protocols where evaluators know that the model is correct most of the time, but not all of the time.
Consolidating and Developing Benchmarking Datasets for the Nepali Natural Language Understanding Tasks
We introduce twelvw new datasets, creating a new benchmark, the Nepali Language Understanding Evaluation (NLUE) benchmark, for evaluating the performance of models across a diverse set of Natural Language Understanding (NLU) tasks. The added tasks include single-sentence classification, similarity and paraphrase tasks, and Natural Language Inference (NLI) tasks. On evaluating the models using added tasks, we observe that the existing models fall short in handling complex NLU tasks effectively.
Development of Pre-Trained Transformer-based Models for the Nepali Language
We collected 27.5 GB of Nepali text data, approximately 2.4x larger than any previously available Nepali language corpus. Leveraging this data, we pre-trained three different models i.e., BERT, RoBERTa, and GPT-2, exclusively for the Nepali Language.
Local Herb Identification Using Transfer Learning: A CNN-Powered Mobile Application for Nepalese Flora
Collected image datasets for local herbs and trained vision models for classification.