>

Best Year Ever

Written by Adam Schepis

Originally Published 2015-10-20 00:00:00 UTC

(leave your thoughts)

Comments on "Best Year Ever"

Back to the article.
>

Don't be a 10x developer. Be a force multiplier

Written by Adam Schepis

Originally Published 2015-07-22 13:15:00 UTC

(leave your thoughts)

Comments on "Don't be a 10x developer. Be a force multiplier"

Back to the article.
>

The Gloves are Off

Written by Adam Schepis

Originally Published 2015-03-25 16:38:00 UTC

(leave your thoughts)

Comments on "The Gloves are Off"

Back to the article.
>

Keeping Pace

Written by Adam Schepis

Originally Published 2014-12-23 13:10:00 UTC

(leave your thoughts)

Comments on "Keeping Pace"

Back to the article.
>

A Fresh Coat of Paint

Written by Adam Schepis

Originally Published 2014-10-09 11:09:00 UTC

(leave your thoughts)

Comments on "A Fresh Coat of Paint"

Back to the article.
>

Integrating Heroku addon SSO in Rails using Devise

Written by Adam Schepis

Originally Published 2012-04-30 14:24:28 UTC

(leave your thoughts)

Comments on "Integrating Heroku addon SSO in Rails using Devise"

Back to the article.
>

Adding to a Devise model after creation (Rails 3.1)

Written by Adam Schepis

Originally Published 2011-09-26 08:36:06 UTC

(leave your thoughts)

Comments on "Adding to a Devise model after creation (Rails 3.1)"

Back to the article.
>

Why I Go Home: A Developer Dad's Manifesto

Written by Adam Schepis

Originally Published 2011-09-15 14:31:54 UTC

(leave your thoughts)

Comments on "Why I Go Home: A Developer Dad's Manifesto"

Back to the article.
>

A Script to Update Riak Config Files

Written by Adam Schepis

Originally Published 2011-08-25 08:42:08 UTC

(leave your thoughts)

Comments on "A Script to Update Riak Config Files"

Back to the article.
>

Break the Build!

Written by Adam Schepis

Originally Published 2011-07-19 13:20:09 UTC

(leave your thoughts)

Comments on "Break the Build!"

Back to the article.
>

Drinking from a Firehose

Written by Adam Schepis

Originally Published 2011-06-27 06:36:24 UTC

(leave your thoughts)

Comments on "Drinking from a Firehose"

Back to the article.
>

Setting up a 3 Node Riak Cluster with EC2 Cluster Compute Instances

Written by Adam Schepis

Originally Published 2011-06-06 13:49:02 UTC

(leave your thoughts)

I've been doing some investigation into a few NoSQL databases lately for use in a project at Symantec. One of the NoSQL DBs I really like and wanted to test some specific use cases for is Riak. I already think that Riak is pretty awesome so i'm not going to get into evangelizing. This blog is purely about the test setup I created on Amazon EC2.

For my tests I decided to create a 3 node Riak cluster. I wanted some beefy hardware that would be roughly analogous to the machines we would be putting Riak on in our real datacenters so I opted to use EC2 Cluster Compute instances to reduce latency. The nodes are Quad-XL (because it was either that or a GPU instance, which I don't need.)

So without any further delay… here is how to set up a 3 node Riak cluster on EC2:

Creating the First Node

Start by spinning up an EC2 Cluster Compute Instance:

Step 1 Step 2

Security Groups

One very important part of this process is Security Groups. AWS uses security groups to define who can access your instances, and how. Here is what my Security Group looks like for my Riak cluster:

Step 3

  • TCP 0-65535 (within the Riak Security Group) is the only thing here that i would justify as a hack :) When i was trying to get my cluster to join with just 8099 open it failed. Perhaps somebody from Basho (or a Riak expert) can fill me in on exactly which ports are needed. I'm not too worried about this since its confined to chatter between my Security Group, but I would never have a production deployment like this.
  • TCP 8099 (within the Riak Security Group) This is the port that Riak uses for intra-cluster data handoff
  • 22 (SSH) - So i can get to the box :)
  • 80 HTTP - Not actually necessary
  • TCP 8098 (Public) - exposing the Riak HTTP interface to the world. In a production environment this would probably be restricted to the IP-space of the servers that would be interacting with Riak.

PS – I just learned while doing this that you can define a port to be available to all instances in a security group. How long has that been around?! It is an awesome feature! <3

Installing Riak

Once your instance is up and running, installing Riak is a breeze (much of this is borrowed from the excellent Riak RHEL install documentation). The only thing I had to poke around a little bit to figure out was

wget http://downloads.basho.com/riak/riak-0.14/riak-0.14.2-1.el5.x86_64.rpm sudo yum install openssl098e sudo rpm -Uvh riak-0.14.2-1.el5.x86_64.rpm

Note: I got a few errors from chcon, but they didnt seem to cause any issues.

Okay! We've got Riak installed. Next I edited the Riak app.config file to bind to all IP address, just for external testing:

in /etc/riak/app.config:

{http, [{ "127.0.01", 8098 }]},

becomes

{http, [{ "0.0.0.0", 8098 }]},

Now you can run sudo riak start to start Riak. If your Security groups are set up correctly you should be able to hit it via port 8098.

Now, shut down that instance. In AWS create an AMI from the instance (right click the instance, choose "Create Image (EBS AMI") Once your AMI is created launch two more EC2 instances from the AMI. This spares you the effort of having to install Riak on each node in your cluster. It also makes scaling out easier in the future since you can create a new instance, edit some config (preferably via Puppet or Chef in an production environment) and run riak join to join the cluster.

Once your two instances are up and running, I literally followed the Riak documentation at http://wiki.basho.com/Basic-Cluster-Setup.html. It was really that easy. The was one gotcha, though. Make sure that when you choose the IP address to bind (and name)for the nodes in your cluster that you use the EC2 Private IP Address (or DNS name should be fine too.)

Step 4

I do have some concerns that this will cause problems on reboots, but I haven't tested it yet. Anyone else done this and determined that for sure?

There you have it. At this point you should have a fully functional 3 node Riak cluster running on AWS EC2 Cluster Compute Instances. It should be pretty damn fast! The only thing I added on top of this was an Elastic Load Balancer that would round robin the traffic between the three nodes so that I could test how it is really scaling.

For the record, I have made my AMI public (just a base Riak install.) The AMI ID is: ami-981ee7f1

Comments on "Setting up a 3 Node Riak Cluster with EC2 Cluster Compute Instances"

Back to the article.
>

Nostalgia be damned! I'll never go back to spinning hard drives.

Written by Adam Schepis

Originally Published 2011-05-27 12:54:11 UTC

(leave your thoughts)

Comments on "Nostalgia be damned! I'll never go back to spinning hard drives."

Back to the article.
>

Of Heroes and Pit Crews: Bringing techniques from the surgical world to software

Written by Adam Schepis

Originally Published 2011-05-16 20:10:20 UTC

(leave your thoughts)

Comments on "Of Heroes and Pit Crews: Bringing techniques from the surgical world to software"

Back to the article.
>

Still Bullish on the Cloud

Written by Adam Schepis

Originally Published 2011-05-14 20:33:14 UTC

(leave your thoughts)

Comments on "Still Bullish on the Cloud"

Back to the article.
>

Oops! Got Popular.

Written by Adam Schepis

Originally Published 2011-05-02 10:07:44 UTC

(leave your thoughts)

Comments on "Oops! Got Popular."

Back to the article.
>

I'm Adam, and i'm a recovering Singleton addict.

Written by Adam Schepis

Originally Published 2011-05-02 06:17:32 UTC

(leave your thoughts)

Comments on "I'm Adam, and i'm a recovering Singleton addict."

Back to the article.
>

Productivity: Go for the quick win!

Written by Adam Schepis

Originally Published 2011-04-19 06:18:09 UTC

(leave your thoughts)

Comments on "Productivity: Go for the quick win!"

Back to the article.
>

Development Complexity vs. Operational Complexity

Written by Adam Schepis

Originally Published 2011-04-05 06:43:50 UTC

(leave your thoughts)

Comments on "Development Complexity vs. Operational Complexity"

Back to the article.
>

Local. Ad-hoc. Relevant.

Written by Adam Schepis

Originally Published 2011-03-24 20:57:29 UTC

(leave your thoughts)

Comments on "Local. Ad-hoc. Relevant."

Back to the article.
>

Offline testing of network driven apps

Written by Adam Schepis

Originally Published 2011-03-24 20:27:11 UTC

(leave your thoughts)

Comments on "Offline testing of network driven apps"

Back to the article.
>

Thoughts on Yobongo after a night stranded in SFO.

Written by Adam Schepis

Originally Published 2011-03-17 20:29:20 UTC

(leave your thoughts)

Comments on "Thoughts on Yobongo after a night stranded in SFO."

Back to the article.
>

One More Reason to Fight Against Walled Gardens

Written by Adam Schepis

Originally Published 2011-03-11 20:53:58 UTC

(leave your thoughts)

Comments on "One More Reason to Fight Against Walled Gardens"

Back to the article.
>

Taking a screenshot of a section of a UIView

Written by Adam Schepis

Originally Published 2011-03-06 20:32:33 UTC

(leave your thoughts)

Comments on "Taking a screenshot of a section of a UIView"

Back to the article.
>

Handling many UIAlertViewDelegate callbacks in the same class

Written by Adam Schepis

Originally Published 2011-02-28 16:52:54 UTC

(leave your thoughts)

Comments on "Handling many UIAlertViewDelegate callbacks in the same class"

Back to the article.
>

Pure-FTPd + Passive FTP on Amazon EC2

Written by Adam Schepis

Originally Published 2011-02-23 22:36:15 UTC

(leave your thoughts)

Comments on "Pure-FTPd + Passive FTP on Amazon EC2"

Back to the article.
>

Lets try this again...

Written by Adam Schepis

Originally Published 2011-02-23 02:45:15 UTC

(leave your thoughts)

Comments on "Lets try this again..."

Back to the article.