Why Managing Data Scientists Is Different

Successfully managing a data science team requires skills and philosophies that are different from those that arise in managing other groups of smart professionals. It’s wise to be aware of the potential organizational frictions and trade-offs that can crop up.

Roger M. Stein Reading Time: 5 min 

Topics

diverge

The process of managing a data science research effort “can seem quite messy,” writes MIT Sloan’s Roger M. Stein. That can be “an unexpected contrast to a field that, from the outside, seems to epitomize the rule of reason and the preeminence of data.”

Image courtesy of Flickr user janneke staaks.

While businesses are hiring more data scientists than ever, many struggle to realize the full organizational and financial benefits from investing in data analytics. This is forcing some managers to think carefully about how units with analytics talents are structured and managed.

How can organizations realize the promise of the evolving disciplines that we broadly call analytics?

Although financial firms were among the first to recruit “quants” to use sophisticated mathematical models and high-powered computing hardware, analytics groups have now taken hold in areas ranging from health care to political campaigns to retailing to sports. Organizations like these can benefit from the insights gained by financial service firms on how best to manage teams doing advanced analytics. It requires skills and philosophies that are different from those that arise in managing other groups of smart professionals.

Rather than just involving oversight and planning, managing a data science research effort tends to be a dynamic and self-correcting process; it is difficult to plan precisely either a project’s timing or final outcomes. For those unused to this type of work, this process can seem quite messy — an unexpected contrast to a field that, from the outside, seems to epitomize the rule of reason and the preeminence of data.

Compounding the friction that this uncertainty generates is the highly technical nature of quantitative research, which can strain relationships between data science teams and other business units. In most organizations, the consumers of data mining or analytic modeling are line managers. However, because many of them aren’t trained in data science, many managers aren’t easily able to evaluate the technical details of a project; as a result they aren’t able to judge the quality of the research — or determine whether a project should take as long as it does. The reverse is true as well: Less experienced data scientists sometimes ignore the rich business experience that line managers could offer them and thus miss out on essential insights that would improve the result or shorten the research process.

Topics

About the Author

Roger M. Stein is a senior lecturer in finance at the MIT Sloan School of Management and a research affiliate at the MIT Laboratory for Financial Engineering. He is the former chief analytics officer and senior managing director of product strategy at State Street GX. Before that he held a number of senior positions in industry including president of Moody’s Research Labs and managing director of research and academic relations globally for Moody’s Corp. Portions of this blog post were adapted, with permission, from Active Credit Portfolio Management in Practice by Jeffrey R, Bohn and Roger M. Stein (Wiley, 2009).

Tags:

More Like This

Add a comment

You must to post a comment.

First time here? Sign up for a free account: Comment on articles and get access to many more articles.

Comments (2)
Eric Skarsdale
This article is extraordinary. I've rarely seen such a blatant example of special pleading. "Data Scientists are different. Our managers don't understand what we do. We don't always know how long something's going to take before we start and it's difficult to evaluate the success after we're finished." I'm paraphrasing, of course, but I don't think unreasonably. As if all of these statements don't apply to a huge number of professionals in a vast number of domains.

As for the statement that "Analytic models and machine learning approaches do not take ideological positions", this is just breathtakingly naïve. Anyone with the slightest awareness of the debates around algorithmic bias (e.g. something like this https://www.theguardian.com/technology/2017/apr/13/ai-programs-exhibit-racist-and-sexist-biases-research-reveals) would be embarrassed to write that.

As ever I'm reminded of the old line - "What's the difference between a data scientist and a data analyst? About $50,000 and a sense of entitlement."
Bart Hamers
The usage of scrum project management together with design thinking techniques can ease the QTC dilemm in Data Science projects. Scrum will put extra attention on the added value of more advanced analytics. Design Thinking will simulate the team to include new techniques, while it forces them to test their brilliant ideas to the 'real world' by rapid prototyping.