Justin Lovelace

Gates Hall

Cornell University

Ithaca, NY

My research focuses on building generative AI systems that are efficient, controllable, and grounded by explicitly modeling semantic latent representations. Rather than treating internal representations as implicit and inaccessible, I develop diffusion models that operate directly in these latent spaces, enabling transparent planning, fine-grained steering, and reliable grounding in external constraints across language, speech, and image domains. I am fortunate to be advised by Prof. Kilian Q. Weinberger.

Previously, I earned my M.S. in Language Technologies from Carnegie Mellon University, where I conducted NLP research under the advisement of Dr. Carolyn Rosé. I completed my undergraduate studies at Texas A&M University, where I engaged in Clinical NLP research with Dr. Bobak Mortazavi.

selected publications

ICLR

SpeechOp: Inference-Time Task Composition for Generative Speech Processing

Justin Lovelace, Rithesh Kumar, Jiaqi Su, and 3 more authors

International Conference on Learning Representations (ICLR), 2026

Bib Paper

@article{lovelace2025speechop,
  title = {SpeechOp: Inference-Time Task Composition for Generative Speech Processing},
  author = {Lovelace, Justin and Kumar, Rithesh and Su, Jiaqi and Chen, Ke and Weinberger, Kilian Q and Jin, Zeyu},
  journal = {International Conference on Learning Representations (ICLR)},
  year = {2026},
  paper = {https://arxiv.org/abs/2509.14298}
}

ICLR

Adaptive Moments are Surprisingly Effective for Plug-and-Play Diffusion Sampling

Christian Belardi, Justin Lovelace, Kilian Q Weinberger, and 1 more author

International Conference on Learning Representations (ICLR), 2026

Bib Paper

@article{belardi2026adaptive,
  title = {Adaptive Moments are Surprisingly Effective for Plug-and-Play Diffusion Sampling},
  author = {Belardi, Christian and Lovelace, Justin and Weinberger, Kilian Q and Gomes, Carla P},
  journal = {International Conference on Learning Representations (ICLR)},
  year = {2026},
  paper = {https://openreview.net/pdf?id=qYDObsHldZ}
}

ICLR

Pre-training Limited Memory Language Models with Internal and External Knowledge

Linxi Zhao, Sofian Zalouk, Christian Belardi, and 5 more authors

International Conference on Learning Representations (ICLR), 2026

Bib Paper

@article{zhao2025pretraining,
  title = {Pre-training Limited Memory Language Models with Internal and External Knowledge},
  author = {Zhao, Linxi and Zalouk, Sofian and Belardi, Christian and Lovelace, Justin and Zhou, Jin Peng and Weinberger, Kilian Q and Artzi, Yoav and Sun, Jennifer},
  journal = {International Conference on Learning Representations (ICLR)},
  year = {2026},
  paper = {https://arxiv.org/abs/2505.15962}
}

COLM

Stop-Think-AutoRegress: Language Modeling with Latent Diffusion Planning

Justin Lovelace, Christian Belardi, Sofian Zalouk, and 3 more authors

Conference on Language Modeling (COLM), 2025

Bib Paper

@article{lovelace2025stopthink,
  title = {{Stop-Think-AutoRegress}: Language Modeling with Latent Diffusion Planning},
  author = {Lovelace, Justin and Belardi, Christian and Zalouk, Sofian and Polavaram, Adhitya and Kundurthy, Srivatsa and Weinberger, Kilian Q},
  journal = {Conference on Language Modeling (COLM)},
  year = {2025},
  paper = {https://arxiv.org/abs/2602.20528}
}

ACL Findings

Diffusion Guided Language Modeling

Justin Lovelace, Varsha Kishore, Yiwei Chen, and 1 more author

Findings of the Annual Meeting of the Association for Computational Linguistics (Findings of ACL), 2024

Bib Paper

@article{lovelace2024diffusion,
  title = {Diffusion Guided Language Modeling},
  author = {Lovelace, Justin and Kishore, Varsha and Chen, Yiwei and Weinberger, Kilian Q},
  journal = {Findings of the Annual Meeting of the Association for Computational Linguistics (Findings of ACL)},
  year = {2024},
  paper = {https://arxiv.org/abs/2408.04220}
}

NeurIPS

Latent Diffusion for Language Generation

Justin Lovelace, Varsha Kishore, Chao Wan, and 2 more authors

Conference on Neural Information Processing Systems (NeurIPS), 2023

Bib Paper

@article{lovelace2023latent,
  title = {Latent Diffusion for Language Generation},
  author = {Lovelace, Justin and Kishore, Varsha and Wan, Chao and Shekhtman, Eliot and Weinberger, Kilian Q},
  journal = {Conference on Neural Information Processing Systems (NeurIPS)},
  year = {2023},
  paper = {https://arxiv.org/abs/2212.09462}
}