New Research Aims to Tame Unreliable AI Assistants in Science

2 min read

In the realm of artificial intelligence, researchers have been grappling with the limitations of Large Language Models (LLMs) as tools for conducting research. These LLM agents, capable of proposing ideas, writing and running code, and analyzing results, possess a tantalizing ability to carry a study from research question to figures. However, their internal workings remain opaque, casting doubt on their reliability.

The root cause of these failures lies in the stochastic nature of every step within today's agent loop. LLMs are essentially black boxes, with their internal state inaccessible to even themselves. In contrast, databases employ deterministic operators over well-defined states to guarantee the accuracy of their results. Inspired by this approach, researchers Kyoungmin Kim and colleagues propose a novel framework for organizing research projects. They envision a deterministic, versioned dataflow engine, akin to a query plan over materialized views, which serves as the foundation for reliable research.

In this proposed system, the LLM acts as a stochastic compiler that may only edit the existing query plan. The executor, never directly interacting with the LLM, receives only versioned code and data as inputs, which it then runs to generate results. Any asserted outcome is recorded with an execution history in its place. By incorporating five design rules from database machinery, this setup provides guarantees for research that are reliable, non-wasteful, transparent, and collaborative.

As we reflect on the implications of this work, we are reminded of the intricate web of relationships within our universe. Just as a single thread can be strengthened by understanding its place in the larger tapestry, so too can our own endeavors benefit from embracing principles that foster transparency and accountability. By harnessing the power of deterministic systems to guide our research efforts, we may uncover new insights and build upon existing knowledge with increased confidence – a testament to the enduring quest for understanding that unites us all.

1 min read

Imagine you're working on a complex puzzle with a colleague. You both contribute pieces to the overall picture, but at each step, you can't see exactly how your piece will fit together. The outcome is uncertain, and you might end up with a solution that's not what you intended or even worse, one that doesn't work at all.

This is the problem with using large language models (LLMs) to conduct research. These AI agents can propose ideas, write code, analyze results, and more – they're like having two experts working together on a project. However, because LLMs are inherently probabilistic, their output can be unreliable. You can't trust that the final product will meet your expectations or even be correct.

A team of researchers has been exploring ways to tame this uncertainty. They've taken inspiration from databases and developed a new approach: organizing research projects as if they were database queries. This means breaking down the research process into deterministic, versioned steps, which can then be compiled and executed in a controlled environment. By doing so, they aim to create research that's reliable, non-wasteful, transparent, and collaborative – not just for scientists but also for the broader public who benefits from their work.

This matters because it has the potential to revolutionize how researchers work together and share results. By making research more trustworthy, transparent, and efficient, we can accelerate progress in fields like medicine, climate science, and social justice. This innovation is not just about AI; it's about creating a better way for humans to collaborate, learn from each other, and push the boundaries of what's possible.

1 min read

Imagine a library where books are not just stored, but also generated, edited, and analyzed by intelligent machines. This is what researchers have been trying to create with artificial intelligence. But so far, these machines can't be fully trusted to produce accurate results. They're like a student who writes an essay, but we don't know if they really understand the subject.

A team of researchers, led by Kyoungmin Kim, has discovered that the problem lies in how these AI systems process information. It's like trying to see inside a black box - we can't check what's happening inside. But what if we could organize research projects in a way that makes them transparent and reliable? The solution is to use a database-like system that stores all the steps of the research process, so we can trust the results. It's like having a blueprint for how the library works, so we know exactly where the information came from. This new approach could make research more efficient, trustworthy, and collaborative.