Subject: 'The Carbon Bathtub' is National Geographic Magazine's 'Big Idea' for December

Posted by John Sterman on 11/20/2009
In Reply To:'The Carbon Bathtub' is National Geographic Magazine's 'Big Idea' for December Posted by Tony Phuah on 11/20/2009

Message:

Tony,

I generally use a discovery model, with some direct instruction after they have tried to do some stock-flow examples (both graphical integration and stock-flow mapping) for themselves, and then open ended assignments in which they have to use these skills on multiple examples. We do lots of buzz groups in class ("work with the person next to you for the next 5 min, and draw a causal diagram or stock- flow map of this issue....").

It's important to note that the disappointingly poor results we get with our students on stock-flow reasoning are on the first day of my course, before they've had any exposure to system dynamics. We do this on purpose to capture their ability to understand stock and flow reasoning before they've studied SD. Thes discouraging results have been replicated at many other universities and with other populations; indeed, most of the time these other groups do much worse than the MIT students.

The natural question is whether the students do better after taking the course. I've now done that study, and you can read it here:

http://jsterman.scripts.mit.edu/On-Line_Publications.html#2009training

The brief answer is: yes, the training definitely helps.
Specifically, after a rather short exposure to system dynamics and stock-flow concepts (about 10 classes total, only a few of which cover stocks and flows), there is a large, significant improvement in performance. however, a number still use erroneous pattern matching logic. There weren't any important correlations between performance and participant demographics such as age, prior field of study, country of origin, etc. except for a gender effect (males outperformed females); the reasons for this are unclear, though the paper discusses possibilities.

By the way, you may say, well, this is obvious: of course training improves performance. however, as I show in the literature review section of the paper, among the small number of prior studies that seek to test whether formal training in system dynamics improves people's ability to relate stocks and flows, the only one published in a peer-reviewed journal found no improvement in two of the three experiments they reported. So there's actually not been much, if any, evidence from carefully designed experiments showing that SD training matters.

The paper includes an appendix with the class syllabus and the assignments on stocks and flows.

People are most welcome to use/adapt/improve these materials provided they give the usual attribution to the source.

Thanks

John