The importance of causal reasoning in AI

I think it would be unreasonable to expect causal reasoning to be the key to "true" A.I., but it would certainly be interesting to implement. I remember from many years ago playing with stupid chatbot that had one very interesting feature : it could do crude deductive reasoning, connecting simple, short chains of if-then statements. I imagined that if you could make this sufficiently extended and detailed we'd get something like causal reasoning. Of course it would have to be fantastically complicated to do anything useful, but then, our own understanding of the word is fantastically complex. Surely we reason by a mixture of induction and something at least similar to true deduction. We recognise patterns, but infer and understand (even if incorrectly) deeper underlying causes that allow us to make predictions in unfamiliar situations. I doubt there's any single magic tool we could implement that would give true sentience, but this does seem interesting.

That’s a dramatic thing to say, that science has abandoned cause and effect. Isn’t that exactly what all of science is about?

Of course, but you cannot see this noble aspiration in scientific equations. The language of algebra is symmetric: If X tells us about Y, then Y tells us about X. I’m talking about deterministic relationships. There’s no way to write in mathematics a simple fact — for example, that the upcoming storm causes the barometer to go down, and not the other way around.

Mathematics has not developed the asymmetric language required to capture our understanding that if X causes Y that does not mean that Y causes X. Seeing that we lack a calculus for asymmetrical relations, science encourages us to create one. And this is where mathematics comes in. It turned out to be a great thrill for me to see that a simple calculus of causation solves problems that the greatest statisticians of our time deemed to be ill-defined or unsolvable. And all this with the ease and fun of finding a proof in high-school geometry.

We did not expect that so many problems could be solved by pure curve fitting. It turns out they can. But I’m asking about the future — what next? Can you have a robot scientist that would plan an experiment and find new answers to pending scientific questions? That’s the next step. We also want to conduct some communication with a machine that is meaningful, and meaningful means matching our intuition. If you deprive the robot of your intuition about cause and effect, you’re never going to communicate meaningfully. Robots could not say “I should have done better,” as you and I do. And we thus lose an important channel of communication.
https://www.quantamagazine.org/to-build-truly-intelligent-machines-teach-them-cause-and-effect-20180515/