publications

2024

  1. Needle Threading: Can LLMs Follow Threads through Near-Million-Scale Haystacks?
    Jonathan Roberts, Kai Han, and Samuel Albanie
    arXiv preprint arXiv:2411.05000, 2024
  2. GRAB: A Challenging GRaph Analysis Benchmark for Large Multimodal Models
    Jonathan Roberts, Kai Han, and Samuel Albanie
    arXiv preprint arXiv:2408.11817, 2024
  3. SciFIBench: Benchmarking Large Multimodal Models for Scientific Figure Interpretation
    Jonathan Roberts, Kai Han, Neil Houlsby, and Samuel Albanie
    NeurIPS, 2024
  4. Charting New Territories: Exploring the Geographic and Geospatial Capabilities of Multimodal LLMs
    Jonathan Roberts, Timo Lüddecke, Rehan Sheikh, Kai Han, and Samuel Albanie
    In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024

2023

  1. GPT4GEO: How a Language Model Sees the World’s Geography
    Jonathan Roberts, Timo Lüddecke, Sowmen Das, Kai Han, and Samuel Albanie
    In NeurIPS Foundation Models for Decision Making Workshop, 2023
  2. SATIN: A Multi-task Metadataset for Classifying Satellite Imagery using Vision-language Models
    Jonathan Roberts, Kai Han, and Samuel Albanie
    In ICCV Towards the Next Generation of Compter Vision Datasets Workshop, 2023