Sign up for our newsletter and get the latest big data news and analysis.

How AI Can Create Conflict within Businesses – and What to Do About It

The rise of artificial intelligence (AI) and machine learning doesn’t just have the potential to change the way we do business, it already is and it’s only going to transform it faster in the future. In fact, recent research from Gartner says that by 2020, AI will be a top five investment priority for more than 30 percent of CIOs.

Companies – from early stage startups to mature corporations – know how huge the opportunity with AI is and are using every resource possible to bring AI into their products and services, in order to create exceptional consumer or customer experiences and stay competitive. With the pressure on to deliver the benefits and efficiency AI and machine learning products bring, businesses are rushing to develop and deploy. Typically, hurrying causes organizational issues to bubble up to the surface. In this particular AI race, a new hurdle has emerged: creating harmony between data science teams (the people who analyze data to create solutions, products and more), and product teams (the people who get the products to consumers/customers).

On the surface, data science and product sounds like a perfect match. But, like many relationships, it’s not always easy. There’s often a push or pull between them. Push being when the data science team does the research and development (R&D) and then takes their work to the product team for deployment. Pull is the opposite, when product approaches data science for R&D in order to solve for a particular problem.

Regardless of whether a company’s data science and product teams are pushing or pulling, priorities often shift. Product might come to data science one day with an ask and by the time data science has completed R&D, product’s priorities might have shifted. Or, data science might not have enough evidence to convince product that an idea they have will work. In this scenario, product would need to gather hard evidence through AB testing, but AB testing can require significant engineering resources and take up considerable time, deterring product teams from moving ahead with the test and ultimately the project.

Needless to say, the relationship between data scientists and product teams is known for being a fragile one. Typically, the relationship between these two teams in most organizations is built based on misunderstanding on each other’s processes. For example, data scientists are the company’s explorers – they live and breathe data, form hypotheses and try to prove their theories. Dead ends are to be expected as that’s a part of their role.

Conversely, product teams are like mini product CEOs. At the highest level, they are responsible for the product lifecycle, from conceptualization to strategy to timely deployment. There are a variety of dynamics and layers involved in this process including consumer and market research, test results, changing business goals, shifting priorities and deadlines and more. The success of a product team relies on their process and ability to be flexible to move fast in the marketplace to deliver value to their consumers/customers.

So, you can see where the harmony begins to slip.

With AI and machine learning here to stay, here’s how these groups can begin working together, without comprising their values:

  • Establish the why – as with any initiative, when the team understands why they are doing something and what the specific goals are, they are marching in unison toward a common goal. Not understanding why a product is being built or the value it brings to consumers will result in fractions within the two teams and a product that can’t sustain market conditions for long. Let’s take email for example. If data science creates a machine learning model and uses click through rates (CTR) as their metric, they’ll train the model to focus on predictions to increase CTRs; if product is looking more at the relevancy of the content in emails as their metric, there is friction in what the model should be trained to do.
  • Give data science a seat at the product table – when data science is not involved in product development, the work they’re doing becomes similar to backend office tasks. This is where the gap begins and the cracks form. Get the data science team involved from the ideation stage and keep them engaged. When product can share data and challenges in every stage of the product feature rollout, it enables data science to better do their job, exploring and driving solutions.
  • Build machine learning models that show results – product teams are metrics-oriented. While yes, machine learning is still an evolving field, data science can help show the value by speaking the product team’s language. For example, if you’re investing in building a machine learning model to weed out spam, then make sure to reflect pre- and post-data, so you can clearly show the reduction in spam after the rollout of your model. Because models are constantly learning and optimizing, it is also good practice to compare results with every optimization in the model.
  • Create common goals framework – in my experience, this is a critical action that not only results in better collaboration between data science and product, but also leads to inspired creativity and a clearer ability to solve business problems. Doing this also creates an environment of co-ownership, where both teams feel a connection to what they are working on and can envision success.

Both of these teams are always working toward the same goals: ship a product that solves specific business challenges and delivers a value. For AI and machine learning products to have a real chance for both businesses and their customers, they need to improve experiences and make life easier, while also ensuring consumers feel confident and safe in everything they’re doing. To achieve that, product and data science need to create a working framework that not only enables collaboration, but also maintains the strengths and values each of these teams bring to the organization.

About the Author

Deep Varma is VP of Engineering at Trulia.Deep leads all engineering functions across the Trulia business as Vice President of Engineering. This includes the front-end teams responsible for building Trulia’s consumer products for mobile, web, and email and other communications, as well as the back-end teams focused on analytics, data science, data warehouse, listings and public records acquisition, personalization, search and QA. During his more than 20 years of experience, Deep has focused on building large-scale distributed web, mobile and data platforms with IBM, ABB, Yahoo! and two successful startups.

 

Sign up for the free insideBIGDATA newsletter.

Leave a Comment

*

Resource Links: