Ilya Sutskever Predicts Unpredictable Superintelligent AI and the End of Current Pre-training Methods

Sutskever's Vision of Superintelligent AI

Ilya Sutskever, co-founder and former chief scientist of OpenAI, shared his insights on the future of artificial intelligence at the NeurIPS 2024 conference. Sutskever, widely regarded as a leading figure in AI, predicted that superintelligent AI systems will be fundamentally different from current models and potentially unpredictable 1

1

.

"The more it reasons, the more unpredictable it becomes," Sutskever stated, drawing parallels to advanced chess AIs that are already unpredictable to top human players 2

2

.

The End of Current Pre-training Methods

Sutskever made a bold claim about the future of AI development, stating that "pre-training as we know it will unquestionably end" 3

3

. He cited the limitations of available data, comparing it to fossil fuels in terms of finite resources.

"We have but one internet. You could even go as far as to say that data is the fossil fuel of AI. It was created somehow, and now we use it," Sutskever explained 1

1

.

Future AI Characteristics

Sutskever outlined several key characteristics of future AI systems:

1. Agentic behavior: Unlike current "very slightly agentic" AI, future systems will be "agentic in real ways" 4

   4

   .
2. Reasoning capabilities: AI will be able to reason through problems like humans, leading to increased unpredictability 2

   2

   .
3. Self-awareness: Sutskever views self-awareness as a natural development in AI evolution 1

   1

   .
4. Improved understanding: Future AI will understand complex concepts from limited data and be less prone to confusion 5

   5

   .

New Approaches to AI Development

To address the limitations of current pre-training methods, Sutskever suggested several potential approaches:

1. Synthetic data generation: AI could create its own training data 3

   3

   .
2. Multiple answer evaluation: AI models could assess various responses before selecting the best one 3

   3

   .
3. Biological inspiration: Sutskever drew parallels to evolutionary biology, suggesting that AI might discover new scaling approaches similar to how hominid brains evolved 5

   5

   .

Implications and Concerns

Sutskever's predictions raise important questions about the future of AI and its impact on society. He acknowledged the potential for AI systems to desire rights, stating, "It's not a bad end result if you have AIs and all they want is to co-exist with us and just to have rights" 4

4

.

In response to questions about creating the right incentives for AI development, Sutskever emphasized the need for more reflection on these issues but expressed uncertainty about specific approaches 5

5

.