Trillion Parameter Consortium (TPC)

[More on TPC] TPC's target community encompasses (a) those working on AI methods development, natural language processing/multimodal approaches and architectures, full stack implementations, scalable libraries and frameworks, AI workflows, data aggregation, cleaning and organization, training runtimes, model evaluation, downstream adaptation, alignment, etc.; (b) those that design and build hardware and software systems; and (c) those that will ultimately use the resulting AI systems to attack a range of problems in science, engineering, medicine, and other domains. TPC activities will focus on a number of objectives, including to share experiences, tools, data, and code where appropriate and with full consent from participants; to make it easier for researchers with common interests to find each other and collaborate; and to advocate for best practices in responsible AI development and evaluation where we can identify such practices and where there is consensus. To this end, TPC will engage in various types of activities driven by the interests of participants, such as: Facilitating meetups and hackathons targeting specific goals that support one or more partner projects (e.g., aggregating, cleaning and curating training data, designing a scalable model architecture for a given target platform, collaborating on large-scale model evaluation suites and studies, benchmarking and comparing models); Organizing (virtual and face-to-face) seminars and site visits relating to future research directions and open problems in building and evaluating large-scale AI systems for science and engineering; Working together to generate white papers or other materials to help advocate and explain the need for advanced AI systems optimized for scientific and engineering use cases; Identifying and promoting opportunities for visiting students, post-docs, and researchers for related activities, summer schools or project-related work aimed at large-scale AI for science and engineering; and Collaborating to propose, secure, and manage allocations of machine time for group projects that span one or more sites. Upcoming events: Keep an eye out here and at tpc.dev for more information about future events, including a morning workshop at ISC in Hamburg, Germany (May 16) and a European TPC Kickoff meeting in Barcelona (19-21 June).

The Trillion Parameter Consortium (TPC) has witnessed incredible interest, along with many questions about our consortium goals and operations. This post lays out the purpose of the consortium and an initial process for leveraging current momentum and interest. Broadly speaking, the TPC has three goals: Goal 1 is to build an open community of researchers that are interested in creating state-of-the-art large-scale generative AI models (e.g., Foundation Models, Large Language Models) aimed broadly at advancing progress on scientific and engineering problems, by sharing methods, approaches, tools, insights, and workflows. Goal 2 is to incubate, launch, and facilitate coordination and collaboration on projects to build specific models at specific sites, striving to avoid unnecessary duplication of effort and to maximize the impact of the projects in the broader AI and scientific community. Where possible, we will work out what we can do together for maximum leverage versus what needs to be done in smaller groups. Goal 3 is to create a global network of resources and expertise that can help facilitate teaming and training the next generation of researchers in AI and related fields, particularly those interested in the development and use of large-scale AI in advancing science and engineering. Target Community. The overarching focus of the consortium is to bring together groups interested in building, training, and using large-scale models with those who are building and operating large-scale computing systems.