October 16, 2024
Large Language Models provide a pre-trained foundation for training many interesting AI systems. However, they have many shortcomings. They are expensive to train and to update, their non-linguistic knowledge is poor, they make false and self-contradictory statements, and these statements can be socially and ethically inappropriate. This talk will review these shortcomings and current efforts to address them within the existing LLM framework. It will then argue for a different, more modular architecture that decomposes the functions of existing LLMs and adds several additional components. We believe this alternative can address many of the shortcomings of LLMs. We will speculate about how this modular architecture could be built through a combination of machine learning and engineering.
Dr. Dietterich (AB Oberlin College 1977; MS University of Illinois 1979; PhD Stanford University 1984) is Distinguished Professor Emeritus in the School of Electrical Engineering and Computer Science at Oregon State University. Dietterich is one of the pioneers of the field of Machine Learning and has authored more than 220 refereed publications and two books. His current research topics include robust artificial intelligence, robust human-AI systems, and applications in sustainability.
Dietterich is the 2024 recipient of the IJCAI Award for Research Excellence. Dietterich has also devoted many years of service to the research community and was recently given the ACML Distinguished Contribution and the AAAI Distinguished Service awards. He is a former President of the Association for the Advancement of Artificial Intelligence and the founding president of the International Machine Learning Society. Other major roles include Executive Editor of the journal Machine Learning, co-founder of the Journal for Machine Learning Research, and program chair of AAAI 1990 and NIPS 2000. He currently oversees the Computer Science categories at arXiv.