Rails GemStone/S
Provided By Stores Provided By Stores
Storage
Engine
MySQL objects mapped to relational tables "Stone" object store Smalltalk objects
Memory
Cache
memcached objects marshalled to strings Shared page cache Smalltalk objects
Worker
Process
MRI/Mongrel Ruby objects "Gem" Smalltalk VM Smalltalk objects
Rails "GemStone/R"
Provided By Stores Provided By Stores
Storage
Engine
MySQL objects mapped to relational tables "Stone" object store Ruby objects
Memory
Cache
memcached objects marshalled to strings Shared page cache Ruby objects
Worker
Process
Ruby/Mongrel Ruby objects "Gem" Smalltalk VM Ruby objects
So there you have it: GemStone, it's like Rails, but faster and easier. If only it ran Ruby...
GemStone/S is the state-of-the-art platform for developing, deploying, and managing scalable, high-performance, multi-tier applications based on business objects.

GemStone/S

Transparent Object Persistence

I've often wished that I could just do something where I just change a bunch of models and at the end of the request cycle something magical goes through and collects up all the models that have changed and persists them, and I haven't really gotten to that level yet. Sometimes I feel that ActiveRecord is holding me back from that.

Avdi Grimm

The Punchline

You can just switch to Java, or .NET.

David Laribee

Waa waaah

Persisting objects is hard

Let's go shopping!

Shoehorn

users

id name birthdate email ...
1 Jesse Cooke 9/28/1981 jesse@jc00ke.com ...
2 Jon Doe 6/23/1972 jon@doe.name ...

Shoehorn

addresses

id user_id street_address city ...
1 1 123 Main St Portland ...
2 777 321 Smith Rd Denver ...

Persistence in OO

Supposed to be simple

And it was... in Smalltalk

The Image

plasma lamp

Currently

We store objects as

  • Rows & Columns (with types)
  • BSON/JSON (with types)
  • Strings #yuck

Objects!


Widget.find_by_name("foo")
# or
User.create({ email: "foo@bar.com" })
  

Some Query Language :(


select * from widgets where name = 'foo';
# or
insert into users...
  

SQL - The Leaky Abstraction ™


Client.where("orders_count = ? AND locked = ?",
             params[:orders],
             false)
  

PORO

Plain Old Ruby Object


class Post < ActiveRecord::Base

end
  

Object-relational impedance mismatch

Wikipedia

Agile Data

Cunningham & Cunningham

Coding Horror

Time traveller

The future of Ruby is here, it’s just that we have it over in the Smalltalk world and you guys should come and get it.
Avi Bryant

Avi Bryant

Simple Example


# VM #1
PROOT = Maglev::PERSISTENT_ROOT
pierre = Cat.new("Pierre")
pierre.object_id
# => 1234567
PROOT[:pierre] = pierre
Maglev.commit_transaction
  

# VM #2
PROOT[:pierre].object_id
# => 1234567
  

VM Modes

Transient (default)

Persistent

Transient


Maglev.transient do
  # Will only be persisted locally
  # Won't be written to the stone
end
  

maglev-ruby foo.rb
  

Persistent


Maglev.persistent do
  # Will be written to the stone
  # upon commit
end
  

maglev-ruby -Mpersistent foo.rb
  

Transactions

Begin


# Implicit...
Maglev.begin_transaction
  

Abort


# Revert all persisted classes
# and grab fresh view
Maglev.abort_transaction
  

What is happening?


Maglev.abort_transaction
    

Commit


# Attempt to write all changes to persisted
# classes to the stone
Maglev.commit_transaction
  

Maglev.commit_transaction
    

Persistence by Reachability


# VM #1
cats = []
pierre = Cat.new("Pierre")
cats << pierre
PROOT[:cats] = cats

Maglev.commit_transaction
pierre.object_id
# => 9876543
  

# VM #2
persisted_cats = PROOT[:cats]

persisted_cats.first.object_id
# => 9876543
  

Persistence by Reachability


# VM #3
pierre = PROOT[:cats].first
pierre.toys << :ball_of_yarn

Maglev.commit_transaction
  

# VM #4
Maglev.abort_transaction

PROOT[:cats].first.toys.first
# => :ball_of_yarn
  

What can you (easily) persist today?

Arrays

Sets

Hashes

Counters

Sorted Sets

Common theme

Serialization

Expensive (mentally & CPU cycles)

Something more... exotic

k-D Tree
Bloom Filter
Judy Array

Leftist Tree

Mismatch

  • It's not just about the objects, but the relations
  • It's the graph!

Clojure!

The only nice thing I'll say about databases, as they exist today, or about Java, or other OO languages... the only nice thing I'll say is about transactions. Transactions are awesome. Transactions are composable. We can understand how to reason about them. They're part of what give databases their greatness. So I've got no qualm with transactions.

An example

Blog


class MaglevBlogApp < Sinatra::Application
  # ...

  before do
    @blog = PROOT[:maglev_blogs].first
    @posts = @blog.posts
  end

  get "/" do
    erb :index
  end
end
  

class MaglevBlogApp < Sinatra::Application
  # ...
  get "/posts/:id" do
    @post = @posts.detect { |post|
      post.slug == params[:slug]
    }
    erb :post
  end
end
  

class MaglevBlogApp < Sinatra::Application
  # ...

  post "/post" do
    post = Post.new(params[:title], params[:body])
    @blog.posts << post
    redirect "/"
  end
end
  

class MagLevTransactionWrapper
  def initialize(app); @app = app; end

  def call(env)
    begin
      Maglev.abort_transaction
      status, headers, body = @app.call env
      [status, headers, body]
    ensure
      commit_if_cool status
    end
  end
end
  

class MagLevTransactionWrapper
  # ...

  private
  def commit_if_cool(status=0)
    if (200..399).include? status
      Maglev.commit_transaction
    end
  end
end
  

class MaglevBlog
  attr_accessor :posts, :name, :author
  def initialize(name, author_name)
    @posts = []
    @name = name
    @author = Author.new(author_name)
  end
  # ...
end
  

class MaglevBlog
  # ...

  class Post
    attr_accessor :title, :body
    def initialize(title, body)
      @title = title
      @body = body
      post_initialize
    end

    private
    def post_initialize
      @slug = SecureRandom.hex(12)
    end
  end
end
  

class MaglevBlog
  # ...

  class Author
    attr_accessor :name
    def initialize(name)
      @name = name
    end
  end
end
  

# bootstrap.rb

Maglev.persistent do
  load "maglev_blog.rb"
  blog = MaglevBlog.new("RubyConf 2012!", "jc00ke")
  PROOT[:maglev_blogs] << blog
end
  

maglev-ruby -Mcommit bootstrap.rb
  

An example

Worker Queue

World's worst (but easiest) queue


# pseudo
jobs = []
jobs << Proc.new { puts "doing work" }
jobs.shift.call
          

Slightly-better producer


PROOT[:q] ||= []

while true do
  PROOT[:q] << Proc.new { puts "doing work" }
  Maglev.commit_transaction
end
  

Slightly-better worker


while true do
  Maglev.abort_transaction
  work = PROOT[:q].shift
  Maglev.commit_transaction
  work.call if work
end
  

Commit failure?

Slightly-better producer


PROOT[:q] ||= []

while true do
  PROOT[:q] << Proc.new { puts "doing work" }
  Maglev.commit_transaction
rescue Maglev::CommitFailedException
  puts "Couldn't add the job. Retrying..."
  redo
end
  

Slightly-better worker


while true do
  Maglev.abort_transaction

  work = PROOT[:q].shift
  Maglev.commit_transaction

  work.call if work
rescue Maglev::CommitFailedException
  puts "Someone grabbed this job before me. Go grab another."
  redo
end
  

MagLev-Q

https://github.com/jc00ke/maglev-q

What's the point?

Totals grouped by language (dominant language first):
ruby:            34 (100.00%)

Total Physical Source Lines of Code (SLOC)                = 34
Development Effort Estimate, Person-Months                = 0.07
Schedule Estimate, Months                                 = 0.90
Estimated Average Number of Developers (Effort/Schedule)  = 0.08
Total Estimated Cost to Develop                           = $ 776
 (average salary = $56,286/year, overhead = 2.40).
  

Generated using David A. Wheeler's 'SLOCCount'

Leaderboard

100 points

Leaderboard

Username Score
@jc00ke 99
@sproutworx 99
@mrinterweb 98

Leaderboard

Tied score?

Username Score
@jc00ke 100
@sproutworx 100
@mrinterweb 98

Leaderboard

Tied score?

Username Score
@sproutworx 100
@jc00ke 100
@mrinterweb 98

class LeaderboardItem
  include Comparable

  def <=>(other)
    return  1 if score < other.score
    return -1 if score > other.score

    0
  end
end
  

Leaderboard

Username Score Timestamp
@sproutworx 100 12:15:27
@jc00ke 100 12:16:27
@mrinterweb 98 12:15:00

class LeaderboardItem
  include Comparable

  def <=>(other)
    return  1 if score < other.score
    return -1 if score > other.score

    return -1 if timestamp < other.timestamp
    return  1 if timestamp > other.timestamp

    0
  end
end
  

The moral?

  • The logic should be in your app, not in the DB
  • Use the data structures we have...
  • ... or build the ones you want
  • MagLev let's us persist anything, so have fun!

Did I mention...

it's distributed?

Remote Stone

maglev.jc00ke.com


$> cd $MAGLEV_HOME
$> rake netldi:start
startnetldi[Info]: Starting GemStone network server 'gs64ldi'.
startnetldi[Info]: GemStone server 'gs64ldi' has been started.

%> rake maglev:start
startstone[Info]: GemStone server 'maglev' has been started.
  

My laptop


cd $MAGLEV_HOME
rake netldi:start
startnetldi[Info]: Starting GemStone network server 'gs64ldi'.
startnetldi[Info]: GemStone server 'gs64ldi' has been started.

maglev-ruby --stonehost -e 'puts PROOT[:cats].first.name'
Pierre
  

Killer feature

Transparent Object Persistence

Killer feature

Transparent Object Persistence

Distributed Transparent Object Persistence

Motivation?

Have fun w/objects

Learn from the past

Make your life easier

Find me later

I want to test out this distributed thing on a bigger scale

with your help ;)

Questions?

Benchmarks are...

    Fun

    Insightful

    Stupid

Fun

    How fast?

    How many can it handle?

    Know where you stand

Insightful

    What's slow?

    What's not implemented?

    What's broken?

Stupid

    Micro or Macro?

    Applicable?

    Just for bragging rights? :(

My view

If you ignore the Stupid, they are Fun & Insightful.

They can be a good tool to diagnose issues & push the implementation

RBS Results

RBS Results

Developer time is worth more than benchmark speed

Uncle Ben

With great benchmarks come great responsibility.

Reveal.js

HTML Presentations Made Easy

Created by Hakim El Hattab / @hakimel

Themes

Reveal.js comes with a few themes built in:
Solarized

* Theme demos are loaded after the presentation which leads to flicker. In production you should load your theme in the <head> using a <link>.

Pretty Code


function linkify( selector ) {
  if( supports3DTransforms ) {

    var nodes = document.querySelectorAll( selector );

    for( var i = 0, len = nodes.length; i < len; i++ ) {
      var node = nodes[i];

      if( !node.className ) {
        node.className += ' roll';
      }
    }
  }
}
					

Courtesy of highlight.js.