Tom Boates is the founder of Brilliant By Design, a product design consultancy that focuses on helping clients of all sizes integrate design thinking into their existing workflows, and being hands on in the design phase to deliver digital products their customers will want to use.

For six years, Tom was the Vice President of User Experience at RunKeeper where he helped grow the user base from 50,000 to over 30 million users. Tom is a veteran of a handful of early stage startups in the Boston area (nimbit, matchmine, currensee), and has a passion for doing anything he can to help people be smarter and more efficient in their quest to build better products.

Christine Perfetti recently talked with Tom about his lessons learned while at RunKeeper, including some key takeaways for shipping best-in-class products.

Just the Highlights: The Big Interview Takeaways

We conduct these interviews for product teams: product managers, UX professionals, and developers who are responsible for shipping great products. We understand it’s difficult to take time away from your job to digest content. That’s why we’ve highlighted the key insights for you below. We highly recommend you read the entire interview for additional brilliant insights from Tom.

1. To deliver great products, focus is key. “At Runkeeper, when we tried to be something to everyone instead of everything to someone, we started to notice many hiccups.”
   
   
   
2. There were some challenges when moving from a waterfall process to Agile. To get Product, UX, and Development working well together, we hired the right people, understood it’s about people, not process. We tapped into the creativity of all team members.
   
   
   

3. The Runkeeper team found the happy medium between creating an annual product roadmap and learning as they go by having the cross-functional teams focus on general opportunities each quarter.
   
   
   

4. What are Tom’s biggest lessons learned about delivering great products? It’s about empathy. “Teams have to put themselves in the shoes of their users. Even when you think you know a lot, you probably don’t know enough.”
   
   

Full Interview with Tom Boates

Christine Perfetti: Can you tell me about the beginnings of Runkeeper and your role in shipping the product?
Tom Boates: When I first started at RunKeeper in late 2008, it was just the CEO, Jason Jacobs, and a handful of part-time contractors. At first, I knew nothing about RunKeeper. I wasn’t a runner, but it was a great opportunity to be the sole UX resource at a company and my first chance to design for the iPhone.

When RunKeeper first launched, it was all about tracking activities using the built-in GPS. The product evolved over the years to include more health-related things, such as training plans, preparing for races, and offering users a more complete picture of their activity. One of our taglines was, “Track, measure, and improve your fitness.”

Who was your primary target audience?

It always came back to runners. We went through a couple of identity crises along the way, but everyone on the team eventually came around and embraced that running is a big market, which allowed us to focus there.

In our research, one theme that comes up is that the successful teams have learned to focus on a specific audience. Sometimes the tendency is to want to please everyone, but this can often lead to even more challenges. What has your experience been like?
I agree. At RunKeeper, in the times when we were dedicated to satisfying the needs of runners, things were clicking on all cylinders. When we attempted to broaden our audience, we ran into trouble. When we tried to be something to everyone instead of everything to someone, we started to notice many hiccups. RunKeeper is now primarily focused on runners. I believe that’s what keeps everything on track.

What was your biggest accomplishment that contributed to the success of the company?
I am very proud of how I pushed design to be a more integral part of the company’s culture. It was a long journey.

When I arrived, RunKeeper had no designers. In the early years, it was an uphill battle to make sure the product was at a certain quality level. There were many times that the team wanted to ship things that looked awful, and I had to fight for any amount of time to clean it up.

As more and more people highlighted RunKeeper’s UX as a differentiator among a growing market, everyone started to understand the value of good design and how important it was for the success of the company. By the time I left RunKeeper, everyone – not just the UX team members – felt they were a designer and responsible for the user experience.

I couldn’t have achieved this on my own. While I laid down a good base layer in the early years, it wasn’t until I hired a lot of awesome people that we started weaving design through the fabric of our culture. The team did an excellent job of evangelizing and educating on the importance of user experience.

What tactics did you use to demonstrate the importance of product quality?
The first few years were difficult. We were using a waterfall process and I was the only designer. It took three years before we hired another designer for the team.

At the three-year mark, RunKeeper had reached a roadblock in our process. We needed to “grow up” when we reached 20 employees and find something more efficient than the waterfall process we had been using. It was clear that our process wasn’t working at that scale. I primarily credit Drew Condon, the now Head of UX at RunKeeper, who helped set up a process that worked well for us, though there were definitely others involved.

What were the challenges you experienced with waterfall?
Where to start. Early on, I was one designer who completed all of the interaction design, visual design, and front-end web development work and I supported anywhere from two to six engineers each working on a different project or feature who all needed mockups or clean up work to be done. It was impossible to keep up with all of the demands. I didn’t have the time and resources.

There was no time dedicated to talking to users, understanding their problems, and learning from what we were shipping. I was completely inundated with design work, so I didn’t have the time to work on it.

How did you learn about user needs and pain points?
One of the reasons I believe RunKeeper was so successful is that our CEO was conducting user research, but we didn’t realize it at the time. He was talking to people on social media, he was in the support forums, and he had a great pulse into what people’s problems and needs were. Without this hands on user contact, I’m convinced RunKeeper’s waterfall process wouldn’t have worked. In fact it started not working eventually which is when we felt we had to switch something up.

How did the product process evolve from waterfall to agile?
From a UX team perspective, the first step was identifying what roles we needed. We already had an interaction design competency, but it became clear we needed a visual designer.

Once we were fully staffed, we helped the broader product team understand how UX could partner with them. RunKeeper migrated to an agile process, we got Scrum set up for our sprints, and we jumped right into a three-week agile sprint process and formed a couple of teams. It was a little rough at first, but there are (and were) a lot of smart and awesome people at RunKeeper so we took to it pretty quickly.

What challenges did you experience with this move?
I think some of the developers had a tough time with it at first. Some of them had liked the mindset of having a design specification or a set of mockups before they invested the time to build it. The team members who were onboard with the changes and could adapt stuck around, and the ones who preferred the old method didn’t.

Initially, we experienced some friction between Product/Design and Engineering and some developers were fighting the process. It was seen as Product and Design forcing Development to do something they didn’t want to do, when we were really trying to ensure everyone was spending their time on the most useful activities. It eventually got worked out.

This is a big challenge we see for product teams: How to get Product Management, Engineering, and UX to work effectively. What are your recommendations?

There are a number of things we did that worked well:

First, we started hiring the people who were attuned to an agile environment. We talked to candidates about the process and made sure we hired people who were eager to be a part of that.

Second, we realized that, to get a team to work well together, it’s more about people than process. When making the move to agile, we were thrusting people into something that might seem unnatural. Rather than saying things like “this is why you should like this,” we had to have more of a discussion. We said, “Listen, what are your concerns here really?” or “Maybe you don’t like doing this activity. Here’s how the new process can help you.”

Like anything in life, such as relationships, friendships, and business communications, you need to get to the deeper issue. You have to ask the “5 Whys” to understand what the real issue is and how to solve it.

This was a foreign approach. The original culture of RunKeeper was very much a heads down, get it done approach. There wasn’t a whole lot of communicating initially. We all trusted each other and we did great work. The whole aspect of investing time to communicate at that level was foreign to most of the team at the time.

Finally, we focused on cultivating creativity and making sure everyone felt they were part of the creation process. One of the primary things people didn’t like about the process is that there was a perception that designers or product managers were telling them what to do. What led to this discomfort was the lack of documentation with agile. When everyone started focusing more on collaborating and tapping into each team member’s creativity at earlier points in the process, everyone took more product ownership instead of a battle between design and development. It became much more inclusive.

What is the team composition at RunKeeper? Is it cross-functional?
We broke the teams into what we called “Pods,” and yes, it was small, cross-functional teams. The average team had a product manager, an interaction designer, a visual designer, a QA engineer, and 3-4 software engineers. By the end of 2014, we had three pods of all of equal size. The teams were focused on solving different problems.

What was the ratio of developers to designers?
4:2. We had four software engineers and an interaction and visual designer on each team.

What were the roles of team members?

Ideally, we didn’t get into a discussion about roles. It was the team focused on solving the problems together and each person happened to bring a unique skill set to the table to achieve that goal. But at the start, some teams operated better than others.

The basic way we broke down responsibilities was the the PM was responsible for the what, what the opportunity was and who it was for and UX was responsible for the How, how we would solve that problem. This is oversimplified because, when working well, the UX and PMs communicated and bounced ideas off of each other all the time helping each other find the solutions together, but it is the easiest way to give some kind of definition to the roles.

How did you prioritize product priorities? Did you have a roadmap?
This was a sore topic. (laughs)

At the Executive level, there was always a push for some kind of product roadmap, asking for a one-year plan to give the leaders a rough idea of what was going to happen and where we were going. I was on the Executive team and could absolutely understand the rationale.

The pushback from most of the Product team was “No. We are going to learn as we go so we couldn’t possibly tell you what we’ll be doing in a year let alone 3 months.” The concern on the product side was that even though the Executive teams say the roadmap can change, once it’s put out there, it tends to be expected and set in stone and much harder to change mid-flight whether you like it or not. This can yield attachment to things that just don’t make sense to spend time on anymore.

Could you find a happy medium?
Right around the time I left, we found some balance. We got there through good communication, talking about the issues, and finding a middle ground rather than the two parties saying, “No” to each other.

I was on the Executive and Product teams and understood both sides as best I could. The RunKeeper board expected and needed to know where we were going. At the same time, the product team needed the freedom to learn and iterate at a fast pace.

The happy medium was that we started outlining themes instead of getting too into the weeds with features. Each team focused on a general opportunity each quarter. This gave a high-level focus to each pod and opportunities for the board to anchor to. At the same time, it didn’t constrain the teams to only focus on feature-level details.

What types of themes did the pods focus on?
User facing things like improving the experience of training for an upcoming race, or suggesting and measuring progress against a goal, or even finding and building new things with partners that made sense for everyone involved. The user came first.

Before this, the pods were focused on business metrics, but that approach didn’t really work in my opinion because we weren’t focused on the user problem we were trying to solve. We might move a business metric up, but it was unclear if it was helpful for users.

How did you instrument at RunKeeper to analyze usage behavior?
Analytics was a part of every release. We always had stories built for tracking and testing.

It wasn’t always like that. In the early days, we weren’t tracking. But we eventually saw the huge importance to the success of the product and formed a data team dedicated to instrumenting, understanding and analyzing the data, and looking for patterns of behavior. We used many different tools at various times for various reasons, including Localytics, Google Analytics, Survey Monkey, Verify, Tableau and Redshift (to name a few).

There was always an effort to make sure the data was in one place. It was a challenge because different stakeholders had different needs, but each PM was charged with having the information they needed for their themes.

What type of qualitative research did you conduct?
We were conducting usability tests and bringing in 3-5 people every two weeks per pod on average. We had a small room in the front of the office that we turned into a usability lab. Each interaction designer was charged with bringing people into the office to test something conceptual, something they were working on in that sprint, or the app as it existed today.

In addition to the lab, we live streamed sessions to a back room with an Apple TV and invited the development team to watch the tests, sit in, and see what they could learn. This led to the lightbulb moments when you see an actual user and see them failing with something you build. Just like with usage data, seeing real users succeed or fail with something is a great equalizer when it comes to internal debates about how something should or shouldn’t be built in a certain way.

What are the biggest lessons you learned for building great products while at RunKeeper?
First, it’s all about empathy for your users. Teams have to put themselves in the shoes of their users. This is something every team should push for. Even when you think you know a lot, you probably don’t know enough. Always be learning about the user.

Second, you need to view everyone on the team as a designer. I don’t believe in the mentality of the product designer as this “genius” who couldn’t possibly communicate the “why” behind their designs to people who don’t just “get it.” I believe this approach is only safeguarding the designer from discomfort and is more characteristic of someone creating art than designing products.

Design is very personal, so it can be tough to put things out there for critique. However, if you have a good working relationship between design and development, you get to the place where work is the work and you’re all partnering on a solution. The designers have to detach themselves and create a collaborative tone. More importantly, it’s the designer that needs to initiate this interaction and champion it all the way through to success no matter how difficult if they want it to be a reality.