benchmark.rb - a benchmark utility
----------
require "benchmark"
include Benchmark
----------
benchmark.rb provides some utilities to measure and report the times used and passed to execute.
To measure the times to make "a"*1_000_000:
----------
puts measure{ "a"*1_000_000 }
----------
On my machine (FreeBSD 3.2 on P5100MHz) this reported as follows:
----------
1.166667 0.050000 1.216667 ( 0.571355)
----------
The above shows user time, system time, user+system, and really passed time. The unit of time is second.
To do some experiments sequentially, bm is useful:
----------
n = 50000
bm do |x|
x.report{for i in 1..n; a = "1"; end}
x.report{n.times do ; a = "1"; end}
x.report{1.upto(n) do ; a = "1"; end}
end
----------
The result:
----------
user system total real
1.033333 0.016667 1.016667 ( 0.492106)
1.483333 0.000000 1.483333 ( 0.694605)
1.516667 0.000000 1.516667 ( 0.711077)
----------
To put a label in each report:
----------
n = 50000
bm(7) do |x|
x.report("for:") {for i in 1..n; a = "1"; end}
x.report("times:") {n.times do ; a = "1"; end}
x.report("upto:") {1.upto(n) do ; a = "1"; end}
end
----------
The option 7 specifies the offset of each report accoding to the
longest label.
This reports as follows:
----------
user system total real
for: 1.050000 0.000000 1.050000 ( 0.503462)
times: 1.533333 0.016667 1.550000 ( 0.735473)
upto: 1.500000 0.016667 1.516667 ( 0.711239)
----------
By the way, benchmarks might seem to depend on the order of items. It is caused by the cost of memory allocation and the garbage collection. To prevent this boresome, Benchmark::bmbm is provided, e.g., to compare ways for sort array of strings:
----------
require "rbconfig"
include Config
def file
open("%s/lib/ruby/%s.%s/tk.rb" %
[CONFIG['prefix'],CONFIG['MAJOR'],CONFIG['MINOR']]).read
end
n = 10
bmbm do |x|
x.report("destructive!"){
t = (file*n).to_a; t.each{|line| line.upcase!}; t.sort!
}
x.report("method chain"){
t = (file*n).to_a.collect{|line| line.upcase}.sort
}
end
----------
This reports:
----------
Rehearsal ------------------------------------------------
destructive! 2.664062 0.070312 2.734375 ( 2.783401)
method chain 5.257812 0.156250 5.414062 ( 5.736088)
--------------------------------------- total: 8.148438sec
user system total real
destructive! 2.359375 0.007812 2.367188 ( 2.381015)
method chain 3.046875 0.023438 3.070312 ( 3.085816)
----------
To report statistics of sequential experiments with unique label, benchmark is available:
----------
n = 50000
benchmark(" "*7 + CAPTION, 7, FMTSTR, ">total:", ">avg:") do |x|
tf = x.report("for:") {for i in 1..n; a = "1"; end}
tt = x.report("times:"){n.times do ; a = "1"; end}
tu = x.report("upto:") {1.upto(n) do ; a = "1"; end}
[tf+tt+tu, (tf+tt+tu)/3]
end
----------
The result:
----------
user system total real
for: 1.016667 0.016667 1.033333 ( 0.485749)
times: 1.450000 0.016667 1.466667 ( 0.681367)
upto: 1.533333 0.000000 1.533333 ( 0.722166)
>total: 4.000000 0.033333 4.033333 ( 1.889282)
>avg: 1.333333 0.011111 1.344444 ( 0.629761)
----------
CAPTION is a caption string which is used in Benchmark::benchmark and Benchmark::Report#report.
FMTSTR is a format string which is used in Benchmark::benchmark and Benchmark::Report#report. See also Benchmark::Tms#format.
BENCHMARK_VERSION is version string which statnds for the last modification date (YYYY-MM-DD).
----------
benchmark([caption [, label_width [, fmtstr]]]) do |x| ... end
benchmark([caption [, label_width [, fmtstr]]]) do array_of_Tms end
benchmark([caption [, label_width [, fmtstr [, labels...]]]]) do
...
array_of_Tms
end
----------
benchmark reports the times. In the first form the block variable x is
treated as a Benchmark::Report object, which has report method.
In the second form, each member of array_of_Tms is reported in the
specified form if the member is a Benchmark::Tms object. The
last form provides combined above two forms (See EXAMPLE 3).
The following lists the meaning of each option.
A string caption is printed once before execution of the given block.
An integer label_width is used as an offset in each report.
An string fmtstr is used to format each measurement. See format
The rest parameters labels is used as prefix of the format to the value of block, that is array_of_Tms.
----------
bm([label_width [, labels ...]) do ... end
----------
bm is a simpler interface of benchmark.
bm acts as same as follows:
benchmark(" "*label_width + CAPTION, label_width, FMTSTR, *labels) do
...
end
----------
bmbm([label_width]) do |x|
x.item("label1") { .... }
....
end
----------
bmbm is yet another benchmark. This utility function is
provited to prevent a kind of job order dependency, which is caused
by memory allocation and object creation. The usage is similar to
bm but has less options and does extra three things:
GC.start to
prevent the influence of previous job. bmbm is a 2-pass procedure, this is possible. bmbm returns an array which consists of Tms correspoding to each
item.
----------
measure([label]) do ... end
----------
measure returns the times used and passed to execute the given block as a Benchmark::Tms object.
----------
realtime do ... end
----------
realtime returns the times passed to execute the given block.
----------
Benchmark::Report::new([width [, fmtstr]])
----------
Usually, one doesn't have to use this method directly,
Benchmark::Report::new is called by benchmark or bm.
width and fmtstr are the offset of label and
format string responsively; Both of them are used in format.
----------
report(fmt, *args)
----------
This method reports label and time formated by fmt. See format of Benchmark::Tms for formatting rule.
----------
Benchmark::Job::new(width)
----------
Usually, one doesn't have to use this method directly,
Benchmark::Job::new is called by bmbm.
width is a initial value for the offset label for formatting.
bmbm passes its argument width to this constructor.
----------
item(((|lable|))){ .... }
----------
item registers a pair of (label) and given block as job list.
Maximum length of labels in list plus one.
array of array which consists of label and jop proc.
alias to item.
----------
Benchmark::Tms::new([u [, s [, cu [, cs [, re [, l]]]]]])
----------
returns new Benchmark::Tms object which has u as utime, s as stime, cu as cutime cs as cstime, re as real and l as label.
The default value is assumed as 0.0 for u, s, cu,
cs and re. The default of l is null string ("").
----------
tms1 + tms2
----------
returns a new Benchmark::Tms object as memberwise summation. This method and operator / is useful to take statistics.
----------
tms / num
----------
returns a new Benchmark::Tms object as memberwise division by num. This method and operator + is useful to take statistics.
----------
add do ... end
----------
returns a new Benchmark::Tms object which is the result of additional execution which is given by block.
----------
add! do ... end
----------
do additional execution which is given by block.
----------
format([fmtstr [, *args]])
----------
format returns formatted string of self according to a
fmtstr like Kernel::format. In addition, format accepts
some extentions as follows:
If fmtstr is not given FMTSTR is used as default value.
returns user time.
returns system time.
returns user time of children.
returns system time of children.
returns total time, that is utime + stime + cutime + cstime.
returns really passed time.
returns label.
returns a new array as follows
[label, utime, stime, cutime, cstime, real]
same as format(). See also format.
2002-04-25: bmbm(): killed unused parameter @fmtstr (gotoken) 2001-11-26: Time.times renamed Process.times for ruby17 (gotoken#notwork.org) 2001-01-12: made bmbm module func. bmbm return Tms array. 2001-01-10: added bmbm, Job and INSTALL.rb (gotoken#notwork.org) 2000-04-00: report() prints tag before eval block (gotoken#notwork.org) 2000-02-22: report(): measure -> Benchmark::measure (nakahiro#sarion.co.jp) 2000-01-02: bug fix, documentation (gotoken#notwork.org) 2000-01-01: measure can take a tag as opt. (nobu.nakada#nifty.ne.jp) 2000-01-01: first release (gotoken#notwork.org)
Gotoken (gotoken@notwork.org).