He was arguing that a program is a formal way to describe the process of solving a problem, and that this is provides us with a new way to think about the world around us. He was claiming that the development of the craft of programming is similar to the extension of the kinds of things that can be reasoned about is similar to the extensions provided by geometry, algebra, and calculus.
I think one of his reasons for this argument was based on the observation that it can be really difficult to teach students how to solve problems. For example he teaches electrical engineering classes, and though you can show students several simple components and when you ask them to solve a more complex problem using several of the simple components many will end up being stuck.
However if you provide them with a program that shows the process one goes through in trying to analyze a complex circuit, he claimed that it can help the student solve the problem.
I'm not quite sure I believe that part, since in my own experience it can be really difficult to understand a what a program does. However an equation describing an aspect of the world can be difficult to understand as well. Though once comprehended can impart more understanding than a vaguely worded explanation