Computational approaches to the history of science are in the process of establishing themselves among the standard repertoire of tools in the field and we have seen remarkable successes in their application already. Subfields of sociology of science have focused, since long, on quantitative methods such as bibliometrics and scientometrics. More recently, philosophy of science has experienced a shift towards allowing more empirical approaches including large-scale algorithmic analyses of scientific or methodological concepts. Computational tools can not only help reduce the workload in traditional research in these fields but, more importantly, also open up new avenues which to explore would otherwise be hopeless.

Analyses of co-occurrences and word frequencies as well as more advanced techniques such as topic modeling have helped go beyond identifying only structural features of scientific activities and began scratching the surface of semantics. However, a deeper understanding of scientific concepts, the structure of scientific arguments, and the process of knowledge transformation and spread have remained formidable challenges for computational approaches in the mentioned fields.

With the advent of LLMs this might change now. Natural language processing and machine learning have made a spectacular leap forward in their attempt to capture and analyze meaning and grammatical structures of texts. This promises that LLMs can help HPSS researchers meet the aforementioned challenges. However—besides general issues such as opacity, bias and interpretability—the use of LLMs for HPSS is likely to face unique obstacles arising from the specialized nature of scientific language as well as the specific perspectives and objectives of HPSS. It will be the main goal of this workshop to see how, given these obstacles, the most recent advances in LLM development can help overcome long-standing challenges in HPSS.

Accordingly, the workshop will address two key themes, with the goal of synthesizing them over the course of the event. On one hand, contributions should articulate the specific needs and desiderata of HPSS researchers—what they hope LLMs can achieve for their work. On the other hand, the current state of LLM development should be critically examined to determine to what extent these research goals are becoming attainable. Ideally, contributions will address both these objectives, though submissions focused on only one of them are also welcome.

We particularly encourage contributions that focus on:

• Use cases that demonstrate how LLMs can help resolve current issues in HPSS
• Examples of how LLMs allow researchers to ask and answer new types of questions in HPSS
• How new types of sources and data, made analyzable through LLMs, contribute to novel insights in HPSS research

We look for contributions that help resolve questions like these:

• How can LLMs help gain new perspectives on long-standing problems in HPSS such as determining the relevant contexts of knowledge claims, the dynamics of scientific controversies, problems of incommensurability, and generalizability of case studies?
• How can LLMs handle the specialized language of scientific texts, including technical jargon, citations, and mathematical formulas?
• How can LLMs bridge the gap between qualitative and computational methods and help overcome their limitations?
• How can LLMs be integrated into existing theoretical and methodological frameworks in HPSS, or how should these frameworks evolve to accommodate LLM-based analysis?
• How can we evaluate the validity of results generated by LLMs, given their opacity?
• How can LLMs account for the temporal development of scientific language and knowledge over time?

Format and practical information

The workshop will take place from April 2-4, 2025 at Technische Universität Berlin. The program will consist of an invited keynote and contributed short talks (15+10 min) as well as additional sessions for discussions. Attendance (online and on site) will be free and open to the public but registration will be required. Information on this will follow closer to the date.

To contribute a talk, please send an abstract of your planned contribution of 300-600 words by e-mail to arno.simons@tu-berlin.de by December 31, 2024. We encourage every contributor to present on site and to participate in the whole workshop program. In exceptional cases, we will offer the possibility to present remotely.

Participation of underrepresented groups is particularly welcome, and we may be able to offer financial support to cover travel costs for contributing authors in exceptional cases. Please indicate in your submission if you would like to apply for financial support.

We plan to publish the slides, videos, and abstracts on a suitable platform. We also plan to write a report on the workshop and on the perspectives resulting from it.

The workshop is funded by the European Union through the project “Network Epistemology in Practice (NEPI)” (ERC Consolidator Grant, Project No. 101044932). Views and opinions expressed are however those of the organizers only and do not necessarily reflect those of the European Union or the European Research Council. Neither the European Union nor the granting authority can be held responsible for them.