Data Feed Wizard
OVERVIEW
Problem:
As PitchBook's Direct Data business grows, the current data feed creation process is not scalable. It requires resource-intensive coding, manual configuration, and testing. The 3-4 week turn-around time is a bottleneck for Direct Data, which is the fasting growing segment within PitchBook.
Goals:
- Decrease the delivery time of a new data feed
- Free the most development resources from data feed creation tasks
My Role: Product Designer
Team: Product Manager, Back-end Engineers, Front-end Engineer, QA, Direct Data Sales Engineers
Timeframe: 5 months
Tools: Figma, Miro, FTP Server
RESEARCH
To understand what the current problems are with the data feed creation process, I ran interviews with our Sales Engineers, Direct Data Account Manager, Backend Engineer, Project Manager, and QA Analyst.
Affinity Mapping
I used an affinity map to define pain points, gauge what works well currently, and document feature requests. The themes that are starred are pain points, feature suggestions, and gaps in the current process addressable in Data Feed Wizard Items with a pink dot that are out of scope for the MVP release.
Insights
1. Excel Errors
The current spec writing process is prone to human errors. Sales engineers have a master spec that they copy, delete any unnecessary tabs, and copy & paste information into the spec.
If I make a mistake, like copy and pasting data points twice, a QA associate has to let me know and go back and forth, which slows down feed generation
David, Senior Sales Engineer
2. Internal Communication
If there are any mistakes in the spec, the QA team has to clarify with the Sales Engineers. There is often a delay in communication due to time zone differences. This is a huge pain point on the development side since they can not start coding the feed until they have all the requirements.
It takes time to ask the sales team and wait for an answer. It is a pain point to ask sales engineers and they’re blocked.
Tanya, QA
3. Lack of Automation
There is a lot of customization to the scopes. As the direct data business grows, this level of customization is not scalable. Engineers have to hard code every single data feed. The new tool would address this issue by standardizing scopes and the back-end architecture will be prebuilt already.
Scope customization. Still need to code the scope. Do not like it currently. This is a painful thing. Don’t see any reasons for coding if we already have the columns.
Andrey, Engineer
User Flow
I mapped out the user journey for the Direct Data Account Managers, Sales Engineers, and Developers to understand the holistic experience from when a customer engages with an account manager expressing interest in a direct data solution to when a fee is delivered. Elements in purple are feature candidates for Data Feed Wizard and the highlighted items in blue are current processes that the new tool will displace.
EARLY CONCEPTS
After breaking down the insights from user research, I worked with the product manager on early concepts. These lo-fi wireframes were used to discuss functionality and receive feedback from product leadership, sales engineers, and developers. From the feedback we received, we updated the flow in the final design so that delivery information came first in the flow and eliminated the ability to upload a data dictionary until a later phase release.
USER STORY & PRODUCT ROADMAP
Throughout this project, we had a bi-weekly meeting with Devs where we shared an evolving roadmap. Since this roadmap is shared across product leadership who may have a limited understanding of the technical terms, I created a glossary to demystify the technical jargon. The product roadmap is broken into 5 different phases: MVP, Phase 2, Phase 3, Backlog, and Out of Scope. The first row represents general user activities, the second row is breaking down user tasks. The remaining rows are feature considerations both on the front-end and backend.
USABILITY TESTING
We ran several usability tests with Sales Engineers and the Direct Data Director. Updates are listed below.
Upload & Template
Upload Data Dictionary and Creating a new template were eliminated from MVP
Key Insights:
-
Uploading a data dictionary is not too useful since files are often corrupted
-
Templates can be helpful for academic accounts but is not an immediate need
Search Bar
The Data Files header was eliminated to make the search bar more discoverable
Key Insights:
-
The user had trouble finding the search bar
-
It was nested and hidden under Data Files
Trailing Ranges
Trailing ranges were added in the Apply Universe Section
Key Insights:
-
There needs to be more flexibility with x years
-
Clients want specific year ranges
FINAL MVP DESIGNS
Delivery Information
Sales Engineers select who the client is, what delivery method to use, and how often the feed will update.
Define Universe
This is where the search link from the platform can be input. If clients already have a list of firms they want to track, they can select that file in this flow.
Scope of Data
Clients are given a Data Dictionary to fill out what data points they want in the Excel file. Instead of Sales Engineers manually inputting those data points into the excel file, they can select the buckets they want in Data Feed Wizard.
Apply Universe
This standardization of Apply Universe will set the Direct Team up for scalability in the future. Instead of a freeform text field with highly customizable scopes, we are pulling the most widely used scopes from each file/relational file.
Review & Edit
This page consolidates all information so users can review and edit with ease. Users can also directly download the spec if the Data Feed Wizard is unable to general a feed.
NEXT STEPS & PROJECTED BUSINESS IMPACT
Data Feed Wizard was on pause due to lack of resourcing. It has since been picked up by another designer and due to release in 2022. The business impact is a significant reduction in engineering resources and an increase in customer satisfaction as we decrease the turnaround time to build a feed.
Cost
The average marginal cost to build a new data feed will reduce by 80%
Delivery Time
The average data feed delivery time will reduce from 2-5 weeks to one day
Resourcing
90% of development resources freed up from data feed creation tasks and will be reallocated to higher-value Direct Data projects