Commit 7a29338
committed
Save HTTP connection across requests
This influxdb client is not using persistent connections. This causes a
significant performance loss. Below is a benchmark script that writes a
single point to influxdb. Running it takes 44 seconds of time:
$ ruby -Ilib t.rb
Writing 1000 points
user system total real
write_points 19.270388 1.394931 20.665319 ( 44.622170)
This works out to a write speed of about 22 points per second, or about
50ms per point.
This is:
* Pretty slow. If you are generating points quickly maximum resolution
of 50ms is not great. A user may lose data because they cannot sample
quickly enough.
* Pretty inefficient. For each point written the client library must
establish a TCP connection, establish a TLS session, finally write
some data. Any data written may be restricted by the TCP slow-start
windowing algorithm.
Much more ruby code must be run, almost half the time spent is in the
"user" category, time that could be doing anything else in a ruby
application, or time that could be used by other processes.
This commit caches HTTP connections across requests. Running the same
benchmark takes 4.26 seconds:
$ ruby -Ilib t.rb
Writing 1000 points
user system total real
write_points 0.551663 0.084603 0.636266 ( 4.261201)
This works out to a speed of 234 points per second, or about 5ms per
point. Writing points now no longer need to recreate a TCP connection,
renegotiate a TLS session, or be held up by TCP window sizing
limitations.
This is much more efficient in terms of CPU time used per point, instead
of ~46% of time taken occurring in the "user" category, now only 13% of
time is in "user". The balance of the time is now spent waiting for IO
to complete.
Benchmark script:
require "benchmark"
require "influxdb"
n = Integer ENV["N"] rescue 1_000
influxdb =
InfluxDB::Client.new url: ENV["INFLUX_URL"],
username: ENV["INFLUX_USER"],
password: ENV["INFLUX_PASS"],
time_precision: "u"
def write_point counter, influxdb
points = [
{
series: "test",
values: {
counter: counter,
},
},
]
influxdb.write_points points
end
puts "Writing #{n} points"
Benchmark.bm 12 do |bm|
bm.report "write_points" do
n.times do |i|
write_point i, influxdb
end
end
end1 parent a309a32 commit 7a29338
1 file changed
+23
-6
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
43 | 43 | | |
44 | 44 | | |
45 | 45 | | |
46 | | - | |
| 46 | + | |
47 | 47 | | |
48 | 48 | | |
| 49 | + | |
| 50 | + | |
49 | 51 | | |
50 | 52 | | |
| 53 | + | |
| 54 | + | |
51 | 55 | | |
52 | 56 | | |
| 57 | + | |
| 58 | + | |
53 | 59 | | |
54 | 60 | | |
55 | 61 | | |
| |||
58 | 64 | | |
59 | 65 | | |
60 | 66 | | |
| 67 | + | |
61 | 68 | | |
62 | | - | |
63 | | - | |
64 | 69 | | |
65 | 70 | | |
66 | 71 | | |
| |||
130 | 135 | | |
131 | 136 | | |
132 | 137 | | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
133 | 151 | | |
134 | 152 | | |
135 | | - | |
136 | | - | |
137 | | - | |
| 153 | + | |
| 154 | + | |
138 | 155 | | |
139 | 156 | | |
140 | 157 | | |
| |||
0 commit comments