WordPress database error: [Got error 28 from storage engine]

Lean split testing with Nginx and Mixpanel in 5 minutes - Yan Pritzker – Yan Pritzker

Yan Pritzker software entrepreneur, photographer, musician

Blog :: Git Workflows Book :: Dotfiles :: Photography :: About Me

Get the news feed
Get updates by email
Follow me on twitter

hello, i'm yan

I am a photographer, entrepreneur, technologist, guitarist, climber, and telemark skier

This blog is about startups, blogging, Ruby On Rails, virtualization and cloud computing, photography, customer service, marketing, ux and design, git, and lots more.

I am the CTO at Reverb.com - The Marketplace for Musicians. We're hiring web Ruby and iOS developers!

Selected Reading



Reach me at yan at pritzker.ws

Lean split testing with Nginx and Mixpanel in 5 minutes

Posted 18 October 2012 @ 12pm in a/b, lean, mixpanel, nginx, split testing

Lean split testing with Nginx and Mixpanel in 5 minutes

Here’s a dead simple way to set up a split test without using a fancy application server or Rails application.

Let’s say you’re starting out with a new application and you want to test a few versions of the landing page. In the truly lean fashion, you haven’t built any dynamic application backend yet. All you’ve got is two versions of the homepage in static html, using something like MailChimp on the backend to capture your signups.

Take your two versions and place them in a directory, along with whatever assets (css/js/images) go along with them:

  • public_html/welcome.html
  • public_html/hello.html

Let’s also assume that these pages contain some type of signup form that redirects back to:

  • public_html/thanks.html

That way we’ll know the goal is completed.

Notice that I’ve named them in user friendly ways. This will become important as we’ll have our webserver (nginx) redirect to those pages, so users will see them in the url. Why nginx? Because apache is a horrible nightmare to configure.

Now sign up for mixpanel.com and follow their instructions to place a snippet of their javascript into those two pages. Additionally, add an event to the bottom of each page to identify it:

[cc lang="html"][/cc]

[cc lang="html"][/cc]

[cc lang="html"][/cc]

Now, we’ll set up nginx to redirect to one of these pages “randomly”. Since there is no true random capability built in, we’re going to use the last digit of the user’s ip:


server {
listen 80;
server_name website.com;
root /home/someuser/public_html;

location / {
if ($remote_addr ~ "[02468]$") {
rewrite ^/$ http://website.com/welcome.html break;
if ($remote_addr ~ "[13579]$") {
rewrite ^/$ http://website.com/hello.html break;

Now hit your website. If your ip is even, you’ll see the first page, and if it’s odd, you’ll see the second page.

Go to Mixpanel, and create two Funnels. One that starts with your welcome event, and one that starts with the hello event, and the second step in each funnel is the Signup Completed event. Now you have a funnel analysis for each page, and you can compare each to see how it performs.

All that, and without even an application server.