Tooling for informalization, understanding, and communication of AI-generated science and code will become increasingly important

As AI is used more in science, there is strong incentive to improve the generation process: better workflows, models, and user experience; longer-horizon tasks; automated labs; formal verification infrastructure; and more.

In the limit, the bottleneck is likely to become human understanding of what this process produces. A great deal of engineering and investment goes into generation and formal verification, but far less into the “understanding” or “informalization” side. We will need new user interfaces, new social norms, new infrastructure, new ways of assigning credit, and more in that direction.