Portrait
Shaojun Xia
About Me

Hey! My name is Shaojun Xia.

My current research interests focus on the intersection of Image Processing, LLM/VLM, AI Agents, and Representation Learning. I am particularly interested in developing intelligent multimodal systems that integrate language, vision, and autonomous reasoning capabilities. I have collaborated on artificial intelligence research with researchers and engineers at organizations including Google, Meta, Amazon, TikTok, Foxconn, puffo.ai, and AItonomy Foundation. These collaborations have provided me with valuable understanding in connecting academic research with real-world industry trends. At the same time, I also have some work experience with researchers and students in CS at several institutions (US and Europe).

Previously, due to the nature of my research positions, I have been engaged in a range of AI-related interdisciplinary projects, primarily spanning imaging/signal processing, cancer, neuroscience, physical models. To better focus my energy, my research interests have gradually shifted toward general AI problems (mainly in machine learning and computer vision), while still building on the experience and expertise I developed through my previous work.

To date, I have published more than 15 peer-reviewed journal and conference papers, many of which were in JCR Q1 track. I also have several ongoing preprints (under submission or revision) available on arXiv.

I am open to collaborations. For any suggestions, please reach out to me via the email link on the left.

News
2026
Our survey with AItonomy Foundation, A Survey of Closed-Loop World Models for Robot Learning, is now available.
I was invited to serve as a peer reviewer for Artificial Intelligence Review (JCR Q1, IF=19).
Congratulations to our collaborators at Imperial College London for the Gear project, an open general harness optimization framework. The GitHub repo has nearly 200 stars, with a preliminary version available on Hugging Face.
Our repo on harnesses, human-computer interaction, self-evolution, and long-horizon tasks, Awesome-LLM-Agent-Harness, is now ready. A related review paper currently is coming soon.
Our paper on low-level image representations in VLMs is now available on arXiv. Our ongoing work on video representations in VLMs will also be released on arXiv [coming soon].
We explored emerging trends in AI agents with our collaborators and identified several interesting questions to be extended in future full papers. Our initial drafts are listed below:
For our two neuroscience-related AI papers, SpindleSwin-Net has been accepted by Biomedical Signal Processing and Control (JCR Q2, IF=5.7), and SpindleFlexNet is in press.
Our multi-modal hallucination paper has been submitted to Artificial Intelligence Review (JCR Q1, IF=19) and included by a TMLR paper from Alibaba Group US DAMO Academy.
Our VLM memory review is currently being prepared for submission to ACM Computing Surveys (JCR Q1, IF=30) and attracted growing attention, appearing among the top Google search results for “VLM memory”. Read more
Of our two papers on modified VLM in-context learning, T2T-VICL is in press, and Hidden-Shot has been accepted for publication in Pattern Recognition (JCR Q1, IF=9.1).
2025
I was invited by a Reviewer Engagement Specialist from Springer Nature to join its Trusted Reviewer Pool.
I served as a lead reviewer in SC25 (HPC for Machine Learning Track).
Our paper, CRLM-GAN, has been accepted by Cancer Imaging (JCR Q1, IF = 4.8) and been included by a related npj paper.
Our paper, Evaluation of unified harmonization of CT images across multiple tasks: A step towards AI generalizability, has been accepted by Medical Physics (JCR Q1, IF = 3.2) with an oral presentation in SPIE 2025 (San Diego).
Our paper, Rectal-RadioSAM, has been accepted by Physics and Imaging in Radiation Oncology (JCR Q1, IF = 3.2) and been commented by a journal of Royal Australasian College of Surgeons.
Our paper, Cer-ConvN3Unet, has been accepted by European Radiology Experimental (JCR Q1, IF = 4.7) and been included by Japanese Journal of Radiology and a book. It was also featured by ESR Journals for its automated cervical cancer detection and segmentation pipeline using multi-parametric MRI.
Selected Publications

* equal contribution # corresponding author

GEAR: An Open General Evolutionary Architecture for Recursive Self-Improvement of LLM Agent Harness
GEAR: An Open General Evolutionary Architecture for Recursive Self-Improvement of LLM Agent Harness

Yehui Tang*#, Guanqi Zhu*, Shao-Jun Xia*, Xin Chen, Mengzhe Geng, Hongyang Du, Yuhao Tang

Manuscript 2026

GEAR is an open framework for optimizing LLM agent harnesses through evolutionary search and reproducible evaluation. A task agent generates execution evidence, while a meta agent uses evidence across tasks to propose reusable harness modifications. GEAR manages the search, HITCH runs fixed agent configurations and archives execution evidence, and REAR supports post-experiment analysis without modifying evaluation records.

[Paper] [Code]

Geometric Similarity in VLM Low-Level Vision Representations
Geometric Similarity in VLM Low-Level Vision Representations

Shao-Jun Xia*, Huixin Zhang*, Zhen Lei*, Anlan Sun#, Yuner Zhang, Xiaoyang Chen#

arXiv preprint 2026

GeoSim studies the geometry of low-level visual representations across seven VLMs and 24 tasks. Its four-level framework combines global similarity, local geometry, sparse feature decomposition, and topological analysis to characterize shared structure and the limits of agreement across tasks and models.

[Paper] [Code]

Agentic Representation Learning: How Should AI Agents Be Represented and Learned from Black-Box Agents?
Agentic Representation Learning: How Should AI Agents Be Represented and Learned from Black-Box Agents?

Shao-Jun Xia#, Zihua Meng*, Huixin Zhang*, Huangyuan Su, Jiahan Li, Yizhuo He#

Preprints.org preprint 2026

Agentic Representation Learning proposes a conceptual framework for learning reusable behavioral representations of AI agents from limited black-box interaction. The proposed characterizer selects informative probes under a cost budget, aiming to support behavior prediction and downstream agent adaptation.

[Paper]

Collaborative Memory for Multi-Agent VLM Systems
Collaborative Memory for Multi-Agent VLM Systems

Huixin Zhang*, Shao-Jun Xia*#, Di Wang#, Liangxi Liu#, Hainan Xiong, Zihao Wang

arXiv preprint 2026

This conceptual paper proposes shared visual memory that links observations, agent interpretations, and dependent reasoning. It outlines memory hierarchies, selective sharing, and revision mechanisms so collaborators can retrieve missing context and identify conclusions that need review when interpretations change.

[Paper]

Automated Multi-Agent Organization Design: From Architecture Search to Organizational Learning
Automated Multi-Agent Organization Design: From Architecture Search to Organizational Learning

Shao-Jun Xia*, Yuner Zhang*, Huixin Zhang#, Di Wang#, Yifan Jiang, Liangxi Liu#

arXiv preprint 2026

This paper proposes AMOD, treating functional responsibilities as explicit, testable components of multi-agent organization. It outlines execution-driven revision and budget-matched removal, generic-role replacement, and reassignment tests to assess their value, and develops a research agenda for organizational memory and adaptation across tasks and agent populations.

[Paper]

Memory in Vision-Language Models: Taxonomy, Mechanisms, and Applications
Memory in Vision-Language Models: Taxonomy, Mechanisms, and Applications

Shao-Jun Xia, Yizhuo He*, Yuner Zhang*, Yifan Jiang#, Xiaoyang Chen#, Liangxi Liu, Chenwen Luo, Jinbao Wang#

Preprints.org preprint 2026

A survey of memory mechanisms in VLMs organized by four dimensions: temporal scope, storage location, stored representations, and memory operations. It connects memory architectures with evaluation protocols and applications in long-video understanding, multimodal dialogue, and embodied agents.

[Paper] [Code]

SpindleSwin-Net: End-to-End and Flexible Sleep Spindle Detection for EEG Signals Based on Wavelet and Advanced Vision Transformer
SpindleSwin-Net: End-to-End and Flexible Sleep Spindle Detection for EEG Signals Based on Wavelet and Advanced Vision Transformer

Shao-Jun Xia*, Anlan Sun*, Jing Bao, Zhi-Nan Wang, Xiao-Ting Li, Ying-Shi Sun#

Biomedical Signal Processing and Control 2026

SpindleSwin-Net transforms EEG signals into wavelet time-frequency maps and uses a dual-branch Swin Transformer to detect sleep spindles and locate their boundaries. It supports joint classification and coordinate prediction, with single-dataset and cross-dataset evaluation on MASS and DREAMS.

[Paper] [Code]

Hidden-Shot: Towards One-Shot Task Generalization for Low-Level Vision Generalist Models
Hidden-Shot: Towards One-Shot Task Generalization for Low-Level Vision Generalist Models

Shao-Jun Xia#, Xianzheng Ma, Zichong Meng

Pattern Recognition 2026

Hidden-Shot combines implicit visual task features with language-guided prompts injected into a vision generalist's decoder. Its 3C4U and 3C7U evaluations examine one-shot generalization to four or seven unseen tasks, alongside performance on trained tasks.

[Paper]

SpindleFlexNet: Flexible Sleep Spindles Detection for EEG Signals Based on an Adaptive One-Dimensional RetinaNet-Based Framework
SpindleFlexNet: Flexible Sleep Spindles Detection for EEG Signals Based on an Adaptive One-Dimensional RetinaNet-Based Framework

Shao-Jun Xia*, Jing Bao*, Anlan Sun*#, Xiaoyang Chen, Hongjia Liu, Xiao-Ting Li, Ying-Shi Sun#

arXiv preprint 2026

SpindleFlexNet adapts RetinaNet to one-dimensional EEG, using temporal feature pyramids and anchor-based classification and regression to locate multiple sleep spindles. Evaluation uses subject-wise five-fold cross-validation on spindle-containing 15-second segments from MASS and DREAMS.

[Paper]

CRLM-GAN: A Feature-Constrained GAN-Based Deep Learning Framework for Multi-Parametric MRI-Based Segmentation of Colorectal Liver Metastases Before and After Chemotherapy
CRLM-GAN: A Feature-Constrained GAN-Based Deep Learning Framework for Multi-Parametric MRI-Based Segmentation of Colorectal Liver Metastases Before and After Chemotherapy

Shao-Jun Xia*, Hai-Bin Zhu*, Xiao-Lei Gu*, Jing Bao, Anlan Sun, Yong Cui, Xiao-Ting Li, Ying-Shi Sun#

Cancer Imaging 2025

CRLM-GAN combines a UNet++ generator with a ResNet-50 discriminator and multi-scale feature constraints to segment colorectal liver metastases. Experiments evaluate MRI before and after chemotherapy with separate-condition and mixed-sequence training, plus separate training and testing on a complementary CT dataset.

[Paper] [Code]

T2T-VICL: Cross-Task Visual In-Context Learning via Implicit Text-Driven VLMs
T2T-VICL: Cross-Task Visual In-Context Learning via Implicit Text-Driven VLMs

Shao-Jun Xia*, Huixin Zhang*, Zhengzhong Tu#

arXiv preprint 2025

T2T-VICL uses a teacher-distilled student VLM to turn a demonstration from one task and a query from another into implicit text prompts for frozen image-editing VLMs. Experiments span 12 low-level vision tasks, measuring task-aware alignment and image fidelity.

[Paper] [Code]

Rethinking Hallucinations in LLMs, VLMs, and MLLMs: A Cognitive-Inspired Taxonomy and Survey
Rethinking Hallucinations in LLMs, VLMs, and MLLMs: A Cognitive-Inspired Taxonomy and Survey

Shao-Jun Xia*, Yifan Jiang*, Xiaoyang Chen#, Yizhuo He, Zitong Li, Zhongwei Wan, Anlan Sun, Hao Zheng#, Jay Pujara#

SSRN preprint 2025

This survey proposes a cognitive-inspired taxonomy of hallucinations: perceptual misalignment, context drift, and factual miscalibration. It reviews causes, detection methods, evaluation benchmarks, and mitigation strategies across LLMs, VLMs, and broader multimodal systems.

[Paper] [Code]

Evaluation of unified harmonization of CT images across multiple tasks: A step towards AI generalizability
Evaluation of unified harmonization of CT images across multiple tasks: A step towards AI generalizability

Shao-Jun Xia, Liesbeth Vancoillie, Saman Sotoudeh-Paima, Mojtaba Zarei, Fong Chi Ho, Fakrul Islam Tushar, Xiaoyang Chen, Lavsen Dahal, Kyle J. Lafata, Ehsan Abadi, Joseph Y. Lo, Ehsan Samei

Medical Physics 2025

A physics-informed harmonization framework for lung CT was evaluated across multiple imaging tasks, including lung structure segmentation, emphysema quantification, longitudinal emphysema assessment, and lung nodule quantification. Results demonstrate improved consistency across scanner and reconstruction variability, supporting more generalizable AI and quantitative imaging analyses.

[Paper]

Rectal-RadioSAM: Large Model-Assisted Multi-Parametric Magnetic Resonance Imaging Pipeline for Predicting Response to Neoadjuvant Chemoradiotherapy in Rectal Cancer Without Human Intervention
Rectal-RadioSAM: Large Model-Assisted Multi-Parametric Magnetic Resonance Imaging Pipeline for Predicting Response to Neoadjuvant Chemoradiotherapy in Rectal Cancer Without Human Intervention

Shao-Jun Xia*, Zhi-Nan Wang*, Jia-Qi Wu*, Qing-Yang Li, Yan-Jie Shi, Xiao-Ting Li, Xiao-Yan Zhang#, Ying-Shi Sun#

Physics and Imaging in Radiation Oncology 2025

Rectal-RadioSAM combines four fine-tuned MedSAM networks with radiomic feature extraction and XGBoost to predict pathological complete response in rectal cancer. It uses MRI before and after neoadjuvant chemoradiotherapy for automated tumor segmentation and treatment-response assessment.

[Paper] [Code]

Cer-ConvN3Unet: An End-to-End Multi-Parametric MRI-Based Pipeline for Automated Detection and Segmentation of Cervical Cancer
Cer-ConvN3Unet: An End-to-End Multi-Parametric MRI-Based Pipeline for Automated Detection and Segmentation of Cervical Cancer

Shao-Jun Xia*, Bo Zhao*, Yingming Li*, Xiangxing Kong, Zhi-Nan Wang, Qingmo Yang, Jia-Qi Wu, Haijiao Li, Kun Cao, Hai-Tao Zhu, Xiao-Ting Li, Xiao-Yan Zhang#, Ying-Shi Sun#

European Radiology Experimental 2025

Cer-ConvN3Unet combines ConvNeXt-based tumor detection with three parallel DoubleU-Net segmentors for cervical cancer MRI. The pipeline processes DWI, T2WI, and contrast-enhanced T1WI to identify tumor-containing slices and delineate lesions automatically.

[Paper] [Code]

Academic Service
  • Reviewer: Trusted Reviewer for Springer Nature; Reviewer for Elsevier and IOP; Lead Reviewer at SC25 (HPC for Machine Learning Track), NeurIPS.
  • Reviewed Journals (Partial): Artificial Intelligence Review, Pattern Recognition, Scientific Reports, npj Digital Medicine, npj Precision Oncology, Journal of Neural Engineering, Brain Imaging and Behavior, Cancer Imaging.
Honors & Awards
Duke University
  • Travel Award
    2023
Peking University
  • Outstanding Graduate Award
    2023
  • National Scholarship (0.2%)
    2022
  • Merit Student (10%)
    2022