Sign up for our newsletter and get the latest big data news and analysis.

Keys to Managing Real Time Data During COVID-19

In this special guest feature, Ben Schein, VP of data curiosity at Domo, draws on his many years of experience with real time data working at Target during the days of Black Friday to the current day where he works to monitor a new COVID-19 tracker. Ben has 20 years of experience leading business intelligence (BI), analytics and finance teams. He is an expert in user adoption and implementing large-scale BI and analytics initiatives that deliver quantifiable business value. As Vice President of Data Curiosity, he works to spark the fire of data curiosity in enterprises large and small across the world. His regular focus includes exploring data curiosity and innovation with Domo’s 1,800+ customers and bringing the insights back to Domo’s product development teams. Ben holds a bachelor’s degree in Philosophy, Politics and Economics from the University of Pennsylvania and an MBA in Strategy and Finance from the Carlson School of Management at the University of Minnesota.

When I worked at Target, for many years I would get up at 1 AM on Thanksgiving morning and head to a digital war room as Black Friday deals went live on Target digital properties at 2 AM.  A lot of people in the room where making sure that Target’s guests could purchase what they wanted and successfully check out.  I was making sure we had all the data to understand what was going on (usually good news, sometimes bad news if there were any issues).   The data was near real time, it was relevant and it was massive.  While those nights in the war room where in many ways physically and emotionally draining, they were also exhilarating.

When I left Target two years ago, I thought my days of high stress and high adrenaline real time data were likely behind me.  In most cases, B2B just does not move at that same frenetic pace.  Then the COVID-19 pandemic came along and I was thrown back into real time data as I became involved with a live tracker of key COVID-19 related metrics.  The data was messy but important and impactful.  And there were thousands and thousands of eyeballs on the data.  It was like the Thanksgiving/Black Friday war room all over again.

Alerting and Monitoring

Luckily, I could draw on many years of experience with real time data as we monitored the new COVID-19 tracker.  First and foremost, alerting and monitoring needs to play a key role.  As much as possible this should be automated but also helps to have multiple sets of eyes on the data as well.  For the COVID-19 data we are monitoring multiple sources to understand when data may be off and also checking for changes in the spelling or naming of countries which can throw off the live data. 

Transparency

Another guiding principle for me is transparency.  The more transparent you can be in terms of what is happening with the data and communicating that with your audience.  If there is a delay in data upstream it is best to find ways to communicate that information to your end user.  There is nothing more frustrating than looking at data and not knowing whether it can be trusted or acted upon.  When using real time data you not only have to report the “news” (data) but also share news about how you are gathering that data.

Flexibility

Lastly, flexibility needs to be paramount when dealing with the messiness of real time data.  You need to be able to quickly adjust if data has been corrupted or needs to be displayed in a different way.  By its nature, the real time data cannot and will not wait for weeks or months long development cycles.  Every minute of bad data could mean misinformed decisions or loss of trust from your users.

I often get push back from people asking something along the lines of “what are you really going to do with that data?  What decision can be made?”  These are valid questions and having real time data carries with it a responsibility for leaders to not over react to any one metric or point in time.   And of course it is legitimate to be deliberative about what data is made available in real time. However, there is something very powerful and culture changing about being connected to your data (and by extension your customer or constituent or patient) in real time.  Every time I get the chance to work with this type of data (even in an unfortunate situation like the COVID-19 pandemic) I am reminded of the power, responsibility and excitement of real-time data.     

Sign up for the free insideBIGDATA newsletter.

Leave a Comment

*

Resource Links: