Posted February 23, 2015

Uncovering What Your Mobile Customers REALLY Want

Last week, I spoke at eTail West in sunny Palm Springs, California on how we approach learning from our mobile customers. I’ve included the presentation and transcript below.


Slide 1 

Good afternoon! My name is Ryan Marfone. I am the User Experience Manager for Mobile at and today I’m happy to present to you “Uncovering What Your Mobile Customer Really Wants”. 

Slide 2 

Here is our agenda for today. I’m going to talk to you about the great place where I work everyday and what I do there. Then we will dive into what I like to call the “Four buckets of pure gold”.

Slide 3 was founded in 1996 by the great man on the right, Jay Steinfeld. Over this time has become the #1 online retailer for window-coverings in the world. In January of last year, we were acquired by The Home Depot. In addition to, we operate four other home-goods websites.

Slide 4

Our marketing team consists of about 30 mad scientists. My job is to work with these mad scientists to ensure that everything we do on the mobile channel is delivering on our brand promises. We have to be entrepreneurial so any given day we can wear many hats to get the job done. For me specifically, I could spend time working on strategy, design, development, analytics, QA, or testing. It’s not that we can’t afford to have teams dedicated to these functions. In fact, we do have teams dedicated to many of these functions. It’s just that I choose to be as autonomous as possible because I have a desire to learn something new every day. I’ll explain what I mean as we move on.

Slide 5

So this is my recommendation for learning something new about your mobile customers every single day. In fact, most of this presentation would actually apply to learning things about customers in general. To begin… Find the smartest people in the company and befriend them. You want these people on your side because, oftentimes, they can help you find innovative ways to achieve your goals. In fact, befriend people in other groups. Especially those people who work the closest to your customers. Your sales teams, customer service teams, and customer experience analysts. Once you have established your army of friends, you will feel much more comfortable experimenting without the fear of failure. What I mean by “Experimenting without fear of failure” is that you should approach your newfound desire to learn something new everyday as an experiment and to not fear the results because like all scientific experiments, there are NO wrong answers. Only answers themselves. And as long as you are receiving answers, you are learning something new everyday.

Slide 6

Okay you would like to start experimenting without fear of failure to learn what you mobile customers really want. There are 4 things I do at that have helped me do just that. Voice of the customer, A/B Testing, Usability Testing, and diving into Analytics.

Slide 7

Voice of the customer comes in many forms from many channels. One of the channels that we’ve created at is the CEO Letter. It’s exactly what it sounds like. When a customer purchases a product from us we send them a letter from our CEO, Jay, to ask them for feedback. We receive a lot of really good responses. Mostly positive because our team at is absolutely fantastic at delivering great customer experiences. But every now and then we get some not so good feedback and this is where you really start to see trends that can help you learn where you may be able to give customers what they are looking for.

  1. Categorize your customer verbiage.
  2. Discuss what you’ve learned with your army of smart people.
  3. Plan to do something about what you’ve learned.

One thing I’ve witnessed teams do is overlook positive customer feedback. Remember, the reason why I’m sharing our CEO letter with you today is to talk about learning something new everyday. When I was creating this presentation last month I found this response to a CEO letter from one of our customers in my inbox. Let’s read it.

[Read email]

All good right?…..

  1. The customer placed a couple orders from us over a few months.
  2. They were pleasantly surprised that it was just that easy.
  3. They plan to buy from us again in the future.

Wait what?… They plan to buy from us again in the future? But when I joined the company, everyone made it sound like we had lower retention because…well.. people don’t buy blinds very often. Come to find it, we hear this a lot. Maybe our mobile customers would appreciate the ability to be remembered so they don’t have to start from scratch every time. Its these “aha” moments that can lead to a make or break mobile e-commerce experience. Had I not spent the time carefully reading customer verbiage, I probably wouldn’t have learned this anytime soon.

Slide 8

The second thing we do at to learn is A/B Test our ideas. I think most people understand what A/B testing is but you would be surprised at how many people I’ve met who have never executed a test. So let’s talk about that for a moment. Say we have two designs and want to know which one will perform better. Well I’ve been in many meetings in the past where the loudest person in the room will influence the team to choose design “B” because its clearly much better than A because blah blah blah blah… and BLAH. But the reality is that none of us truly represent the customer as much as the customer themselves. So that’s where AB testing comes into play. A/B testing is facilitated by specialty software like Adobe Target or Optimizely. Using one of these tools you can configure a test to send traffic to one design or the other and configure how you are going to measure success. So up here you see two different versions of our mobile product page at For this test, Design A was the control and Design B was the new design that we wanted to test against the control. I setup the test to send 50% of traffic to design A and the other 50% to design B.

As you can see, I changed a couple things in design B like…

  1. The Add to Cart button is now much larger, in a different position, and a different color.
  2. The product image is a little bit larger
  3. The product name is a little bit larger
  4. The social sharing buttons are now included directly on the page.

Slide 9

Fortunately, design B won this test because…

  1. We saw a 100% lift in our call to action.
  2. We saw a 30% lift in purchase conversion across the mobile site.
  3. Social sharing stayed flat (meaning we didn’t break it).

Slide 10

But not all tests win. Most tests actually fail. At we have a very transparent way of demonstrating this and we celebrate the act of experimentation. In fact, “Experiment without the fear of failure” is one of our core values. Also, in our marketing area we have two gigantic test tubes that we had custom made from someone in Austin, TX. What we do is… for every test we run, we drop in a marble. If a test fails and the control design wins, we drop in a clear marble. If a test succeeds and the new design wins, we drop in a colored marble. As you can see, we obviously have more clear marbles than colored marbles in the tubes. But again, just because a test fails, don’t consider the act a failure because there is much to be learned about your mobile customers from tests that fail.

Slide 11

The third method for learning what your mobile customers REALLY want is to observe them using your site, otherwise known as Usability Testing. Since making the product “easy to use” is so valuable to the customer experience, at we offer incentive to our customers to have them come sit with us for an hour so we can watch them using our products. For mobile, that means two cameras. One on the product and one on the test participant. We define a few tasks that we want our participants to act out and we ask them to think aloud as they are interacting with the product so we can try to understand what they are thinking and compare that to what they are doing. The things you learn about your mobile customers by doing this are so grand that you will never question the investment again. It does take time to facilitate these tests but they are worth it.

Slide 12

Analytics are the last method I’m going to talk about today. They are so incredibly instrumental to creating a solid mobile strategy that I wanted to end on this topic. So if you forget the rest of my presentation, the one thing I want you to really take away is the value of capturing analytical data, measuring it, and using real data to shape your mobile strategies. There are many really good analytics tools on the market. At we use Omniture. I’ve created a report that I so cleverly named “My Mobile Report” because I’m super creative like that. I receive this report in my inbox early every morning. This is the first thing I see when I sit down to begin my day. My report contains data that I personally care about like…

  1. What percent of mobile traffic purchased a product.
  2. What percent of mobile traffic ordered a free sample of material.
  3. How much revenue was generated from the mobile channel.
  4. How much time people are spending on the site.
  5. What pages this traffic is hitting while they are on the site.
  6. Where this traffic is coming from (Organic, Paid Search, Social Networks, Email Marketing, etc.).

I feel like this is an appropriate way to kick off each day. It helps me get into the analytical mindset that gives me confidence that WE ARE focusing our mobile investments on the areas that will make interacting with surprisingly easy.

Slide 13

Thank you for joining me today. I want to give a special thank you to eTail for inviting us out to the event this year. If you would like to get in touch, feel free to email me at Also, to learn more about the experiments we are working on at, please check out our newly created website at Thanks again.