2 天前
Intern-S1：一种科学多模态基础模型
Lei Bai, Zhongrui Cai, Maosong Cao, Weihan Cao, Chiyu Chen, Haojiong Chen, Kai Chen, Pengcheng Chen, Ying Chen, Yongkang Chen, Yu Cheng, Yu Cheng, Pei Chu, Tao Chu, Erfei Cui, Ganqu Cui, Long Cui, Ziyun Cui, Nianchen Deng, Ning Ding, Nanqin Dong, Peijie Dong, Shihan Dou, Sinan Du, Haodong Duan, Caihua Fan, Ben Gao, Changjiang Gao, Jianfei Gao, Songyang Gao, Yang Gao, Zhangwei Gao, Jiaye Ge, Qiming Ge, Lixin Gu, Yuzhe Gu, Aijia Guo, Qipeng Guo, Xu Guo, Conghui He, Junjun He, Yili Hong, Siyuan Hou, Caiyu Hu, Hanglei Hu, Jucheng Hu, Ming Hu, Zhouqi Hua, Haian Huang, Junhao Huang, Xu Huang, Zixian Huang, Zhe Jiang, Lingkai Kong, Linyang Li, Peiji Li, Pengze Li, Shuaibin Li, Tianbin Li, Wei Li, Yuqiang Li, Dahua Lin, Junyao Lin, Tianyi Lin, Zhishan Lin, Hongwei Liu, Jiangning Liu, Jiyao Liu, Junnan Liu, Kai Liu, Kaiwen Liu, Kuikun Liu, Shichun Liu, Shudong Liu, Wei Liu, Xinyao Liu, Yuhong Liu, Zhan Liu, Yinquan Lu, Haijun Lv, Hongxia Lv, Huijie Lv, Qidang Lv, Ying Lv, Chengqi Lyu, Chenglong Ma, Jianpeng Ma, Ren Ma, Runmin Ma, Runyuan Ma, Xinzhu Ma, Yichuan Ma, Zihan Ma, Sixuan Mi, Junzhi Ning, Wenchang Ning, Xinle Pang, Jiahui Peng, Runyu Peng, Yu Qiao, Jiantao Qiu, Xiaoye Qu, Yuan Qu, Yuchen Ren, Fukai Shang, Wenqi Shao, Junhao Shen, Shuaike Shen, Chunfeng Song, Demin Song, Diping Song, Chenlin Su, Weijie Su, Weigao Sun, Yu Sun, Qian Tan, Cheng Tang, Huanze Tang, Kexian Tang, Shixiang Tang, Jian Tong, Aoran Wang, Bin Wang, Dong Wang, Lintao Wang, Rui Wang, Weiyun Wang, Wenhai Wang, Yi Wang, Ziyi Wang, Ling-I Wu, Wen Wu, Yue Wu, Zijian Wu, Linchen Xiao, Shuhao Xing, Chao Xu, Huihui Xu, Jun Xu, Ruiliang Xu, Wanghan Xu, GanLin Yang, Yuming Yang, Haochen Ye, Jin Ye, Shenglong Ye, Jia Yu, Jiashuo Yu, Jing Yu, Fei Yuan, Bo Zhang, Chao Zhang, Chen Zhang, Hongjie Zhang, Jin Zhang, Qiaosheng Zhang, Qiuyinzhe Zhang, Songyang Zhang, Taolin Zhang, Wenlong Zhang, Wenwei Zhang, Yechen Zhang, Ziyang Zhang, Haiteng Zhao, Qian Zhao, Xiangyu Zhao, Xiangyu Zhao, Bowen Zhou, Dongzhan Zhou, Peiheng Zhou, Yuhao Zhou, Yunhua Zhou, Dongsheng Zhu, Lin Zhu, Yicheng Zou
摘要
近年来，大量开源基础模型相继涌现，在多个广受关注的领域取得了显著进展，其性能已接近闭源模型。然而，在高价值但更具挑战性的科学专业领域，现有模型仍主要依赖专家模型，或通用基础模型的进展远落后于热门领域，难以有效推动科学研究的变革，导致开源模型与闭源模型在这些科学领域之间仍存在显著差距。为缩小这一差距，并进一步探索通往通用人工智能（AGI）的路径，我们提出Intern-S1——一种具备通用理解与推理能力、并专精于多模态科学数据解析的专用通用模型。Intern-S1是一种多模态专家混合（Mixture-of-Experts, MoE）模型，激活参数达280亿，总参数量为2410亿，基于总计5万亿个token持续预训练，其中来自科学领域的数据超过2.5万亿token。在后续训练阶段，Intern-S1在“InternBootCamp”环境中先后经历离线与在线强化学习（Reinforcement Learning, RL）训练。为此，我们提出“奖励混合机制”（Mixture-of-Rewards, MoR），实现对1000多个任务的并行协同强化学习训练。通过算法、数据与训练系统方面的集成创新，Intern-S1在在线强化学习训练中实现了顶尖性能。在综合评估基准测试中，Intern-S1在通用推理任务上展现出与开源模型相当的竞争力，且在科学领域显著超越现有开源模型，在专业任务中甚至超越了闭源的最先进模型，例如在分子合成路径规划、反应条件预测、晶体热力学稳定性预测等任务中表现优异。相关模型已开源，可通过以下链接获取：https://huggingface.co/internlm/Intern-S1。