Also in the recent and upcoming releases in "causality" department: The Book of Why: The New Science of Cause and Effect (Judea Pearl, Dana Mackenzie) https://www.nytimes.com/2018/06/01/business/dealbook/review-...

Jamie Robins and I are working on a book that provides a cohesive presentation of concepts of, and methods for, causal inference. Much of this material is currently scattered across journals in several disciplines or confined to technical articles. We expect that the book will be of interest to anyone interested in causal inference, e.g., epidemiologists, statisticians, psychologists, economists, sociologists, political scientists, computer scientists...

I'm a bit surprised that there isn't already a "cohesive presentation", both for teaching and as a resource for practitioners. (Of course, that seems to be when books are written, when the need suddenly seems obvious.) Would any scientists, etc. comment on how the topic is currently taught and what reference is used? Or am I misunderstanding the nature of the topic, such as its significance or generality? Has something changed?

Typically, stats seems to have different usage and teaching by field. Psychologists have their flavor, physicists another. Econometricians, epidemiologists, nutritionists, etc, all different. They're pretty much all asking causal questions, but they're coming up with different techniques to solve the specific weird questions that are unique to their field.

Some fields get to randomize everything. Causal inference is is as simple as asking "is this difference real" when the difference in your experiment would be causal by design. This is what is taught as stats 1 to everybody who starts learning stats.

Many fields don't get to randomly do things :(. Maybe you want to know the effect of a disease, but we as a society have decided that harming people to learn for the eventual greater good is bad (hello "rush out the self driving cars!" proponents!). Or maybe randomizing would involve merging a few multi-billion dollar companies and grad students don't have that budget. For whatever reason, you can't intervene in the same way. I give those two examples because epidemiologists and economists have really extended the state of the art of causal inference because their data doesn't look the same as many other fields. Inference is harder (but still possible!).

Those niche developments have been happening for a while, but haven't been cohesively un-niched. Big (non-randomized) data is becoming a big thing and everyone wants to use it to make decisions. Decisions require causal statements and most people only know how to make those statements for cleanly randomized stats 1 stuff. People seem to be picking up on the fact that some of this previously niche stuff is much more broadly useful.

I bet someday we realize that "causality" is about as real as the ether or force at the subatomic level. That is, something that seems fundamental and obvious but is really just wrong.

It may be wrong in the sense that free will is wrong, but a lack of free will doesn't stop me from making decisions. Causality is all about counterfactuals, and the admission is right there in the name counter-factual. We are already aware.

The most popular framework for causality talks about potential outcomes. If I took my blood pressure meds now (all else equal), what would my blood pressure be at noon? If I skipped my meds today (all else equal), what would my blood pressure be at noon? That tells me the casual effect of the meds on my blood pressure today. But I either will or won't take the meds. One of those branches is simply wrong on a fundamental level, like you say. "What would happen" is in some sense meaningless. But it's sure as shit useful to think about, and that's the whole point.

Said differently, even in a toy model of the universe where everything is deterministic and causality is meaningless on a physics level, causality can be usefully defined on a practical level. Counterfactuals are counter factual, but that's ok.

In a toy model, the ether and Aristotlean mechanics worked ok too. I am not saying we should give up the notion altogether, but that causality as we intuit it may not be the best way to understand the universe.