3.3.1 The Shopping Cart A/B Test
Now that we have learned how to perform a hypothesis test, let’s return to Amazon and
look at a couple examples of A/B testing.
So what we're trying
to do is we're
trying to i
...
3.3.1 The Shopping Cart A/B Test
Now that we have learned how to perform a hypothesis test, let’s return to Amazon and
look at a couple examples of A/B testing.
So what we're trying
to do is we're
trying to improve the
experience for our customers.
That's part of our
charter, part of our values
is to be the most
customer-centric company
in the universe.
So what does it mean to make
things better for a customer?
To me as an economist,
how I'd view
it is if you buy
more of it, that
might be in many,
many instances,
an indication that
you like it more,
that we've improved
things for you.
If you convert
more often, that's
an indication that we've
improved the experience
for our customers.
We can't go out and ask them
questions directly very often
about do you like
this versus that.
We have to look
at their actions.
So the types of metrics that
are the simplest and the most
powerful for us are
how much did you spend,
how many units did you buy.
Do we make a distinction between
the dollar or euro or yen sales
of product and the
units that we sell?
Because we have
product that is at so
many different price points
from a $20 book to a $1,000 TV
and everything in between?
And so it's helpful for us
to look at both the units
that we sell, which is a
measure of the engagement
that we have with customers,
and the absolute dollar sales,
because of course the
impact of buying $1,000
TV may tell us more about your
long-term engagement with us
than one purchase of a $20 item.
The graphic that we decided
to operate on in this A/B test
is the persistent
navigation that we
have at the very top of
every amazon.com page.
It includes things like Welcome.
It includes whether
you're a Prime member.
It includes our logo.
It includes Help.
It includes a link
to your account.
And it includes a little
shopping cart icon that
for a long time was static.
It just looks like a
shopping cart that you
push through the aisles.
If you click on
it, it's the place
that you go to see what you've
put into your shopping cart
during that session
or some other session.
As long as you
remain logged in, we
will persist what's in
your cart, of course.
And you can see the
things that you've
added yesterday, the day
before, and in today's shopping
session.
We find people use it for all
kinds of different things.
The team that built thi
[Show More]
