The New Yorker reports
[B. F.] Skinner was enthralled. Two years after reading the Times Magazine piece, he attended a lecture that Pavlov delivered at Harvard and obtained a signed picture, which adorned his office wall for the rest of his life. Skinner and other behaviorists often spoke of their debt to Pavlov, particularly to his view that free will was an illusion, and that the study of human behavior could be reduced to the analysis of observable, quantifiable events and actions.I realize that people doubted free will before this. Wikipedia says Bertrand Russell rejected free will along with Christianity as a teenager (in around 1890), but also says that he was a big advocate of freedom of thought all his life. I don't know how he reconciles that.
But Pavlov never held such views, according to “Ivan Pavlov: A Russian Life in Science” (Oxford), an exhaustive new biography ...
Pavlov’s research originally had little to do with psychology; it focussed on the ways in which eating excited salivary, gastric, and pancreatic secretions. ... That research won him the 1904 Nobel Prize in Physiology or Medicine. But a dog’s drool turned out to be even more meaningful than he had first imagined: it pointed to a new way to study the mind, learning, and human behavior. ...
The Soviets came to regard Pavlov as a scientific version of Marx. The comparison could not entirely have pleased Pavlov, who rebelled at the “divine” authority accorded Marx (“that fool”) and denied that his own “approach represents pure materialism.” Indeed, where others thought that the notion of free will would come to be discarded once we had a full understanding of how the mind worked, Pavlov was, at least at times, inclined to think the opposite. “We would have freedom of the will in proportion to our knowledge of the brain,” he told Gantt in 1927, just as “we had passed from a position of slave to a lord of nature.”
That year, Stalin began a purge of intellectuals. Pavlov was outraged. At a time when looking at the wrong person in the wrong way was enough to send a man to the gulag, he wrote to Stalin saying that he was “ashamed to be called a Russian.”
Skinner was voted in 2002 as the most influential psychologist of the 20th century. He denied free will. His biological determinism was based on behavior, while others focused on DNA and genes. Marxism was not necessarily opposed to free will, but very much advocated a sort of historical determinism. They all seemed to think that XX century science is going to make humanity so predictable that there will be no room left for free will.
I am just trying to understand what would lead leftist Harvard professors to reject free will. This is just a clue, and I am sure it is an incomplete explanation.