Building Enumerables and Going Lazy
Enumerable is a module with about sixty methods, all built on one you supply.
Write each, include the module, and your own type gains the entire vocabulary.
class Playlist
include Enumerable
def initialize(tracks) = @tracks = tracks
# This is the only method Enumerable needs.
def each(&block)
@tracks.each(&block)
self
end
end
playlist = Playlist.new([
{ title: "Aria", seconds: 210 },
{ title: "Bloom", seconds: 185 },
{ title: "Cinder", seconds: 320 },
])
puts playlist.map { |t| t[:title] }.inspect
puts playlist.select { |t| t[:seconds] > 200 }.map { |t| t[:title] }.inspect
puts playlist.sum { |t| t[:seconds] }
puts playlist.min_by { |t| t[:seconds] }[:title]
puts playlist.sort_by { |t| -t[:seconds] }.first[:title]
puts playlist.each_slice(2).to_a.length
puts playlist.include?({ title: "Aria", seconds: 210 })
Open this example in the editor and change it. Nothing you type is saved.
Sixty methods for the price of one. Add <=> and include Comparable on the
elements and sort, min and max work without a block as well.
Returning an Enumerator when no block is given
The convention every core method follows: with a block, do the work; without one, hand back an Enumerator so the caller can chain.
class Countdown
include Enumerable
def initialize(from) = @from = from
def each
# This one line is what makes countdown.each.with_index work.
return to_enum(:each) { @from } unless block_given?
@from.downto(1) { |n| yield n }
end
end
countdown = Countdown.new(5)
countdown.each { |n| print n, " " }
puts
enumerator = countdown.each
puts enumerator.class
puts enumerator.size # answered by the block passed to to_enum
puts enumerator.next
puts enumerator.next
puts countdown.each.with_index(1).map { |n, i| "#{i}:#{n}" }.inspect
Open this example in the editor and change it. Nothing you type is saved.
Enumerator, and generating values on demand
Enumerator.new takes a block that is handed a yielder. Nothing inside runs
until a value is asked for, so an infinite sequence is perfectly safe.
naturals = Enumerator.new do |yielder|
n = 1
loop { yielder << (n += 1) - 1 }
end
puts naturals.take(5).inspect
puts naturals.first(3).inspect
fibonacci = Enumerator.new do |y|
a, b = 0, 1
loop do
y << a
a, b = b, a + b
end
end
puts fibonacci.take(10).inspect
puts fibonacci.lazy.select(&:even?).first(5).inspect
# External iteration: next pulls one value at a time and remembers where it is.
pages = fibonacci.each_entry
puts pages.next
puts pages.next
puts pages.peek
puts pages.next
Open this example in the editor and change it. Nothing you type is saved.
Lazy, and why the order of the chain matters
A normal chain builds a full intermediate array at every step. lazy builds
none: each element travels the whole chain before the next one starts.
# Without lazy this never finishes: map would try to build an infinite array.
result = (1..Float::INFINITY)
.lazy
.map { |n| n * n }
.select { |n| n % 3 == 0 }
.first(4)
puts result.inspect
# The saving is visible when you count the work.
seen = 0
answer = (1..1_000_000)
.lazy
.map { |n| seen += 1; n * 2 }
.select { |n| n % 7 == 0 }
.first(3)
puts answer.inspect
puts "examined #{seen} of a million"
eager_seen = 0
(1..1_000_000).map { |n| eager_seen += 1; n * 2 }.select { |n| n % 7 == 0 }.first(3)
puts "eager examined #{eager_seen}"
Open this example in the editor and change it. Nothing you type is saved.
force or first(n) ends a lazy chain. Until one of those is called, nothing
has happened at all.
chain = (1..20).lazy.map { |n| puts " computing #{n}"; n * 10 }.select(&:positive?)
puts "nothing has run yet"
puts chain.first(2).inspect
Open this example in the editor and change it. Nothing you type is saved.
Generators that free the memory
The real use is a source too big to hold: a file, a paginated API, a long query. Here is the shape, with an array standing in for the remote source.
PAGES = {
1 => %w[alpha bravo charlie],
2 => %w[delta echo foxtrot],
3 => [],
}
def fetch_page(number)
puts " fetching page #{number}"
PAGES.fetch(number, [])
end
def all_records
Enumerator.new do |y|
page = 1
loop do
rows = fetch_page(page)
break if rows.empty?
rows.each { |row| y << row }
page += 1
end
end
end
# Only page one is ever fetched, because two matches were enough.
puts all_records.lazy.select { |r| r.length == 5 }.first(2).inspect
puts "---"
puts all_records.to_a.length
Open this example in the editor and change it. Nothing you type is saved.
each_with_object, and the reduce trap
words = %w[apple avocado banana blueberry cherry]
# reduce must return the accumulator every time. Forget that and you get nil.
puts words.reduce(Hash.new(0)) { |acc, w| acc[w[0]] += 1; acc }.inspect
# each_with_object hands the accumulator back for you, so it cannot be dropped.
puts words.each_with_object(Hash.new(0)) { |w, acc| acc[w[0]] += 1 }.inspect
# Note the argument order is reversed between the two. It is the most common
# reason one of them "does not work".
puts words.each_with_object([]) { |word, acc| acc << word.length }.inspect
puts words.reduce([]) { |acc, word| acc << word.length }.inspect
Open this example in the editor and change it. Nothing you type is saved.
What to take away
- Define
each,include Enumerable, and sixty methods arrive. - Return
to_enum(:each)when no block was given, so callers can chain. Enumerator.newproduces values on demand; infinite sequences are fine.lazypasses one element through the whole chain before starting the next.- A lazy chain does nothing until
firstorforce. each_with_objecttakes(element, accumulator);reducetakes the reverse.