Adding Benchmarks - Part II

Metrics Recap

To reiterate, we will be using the following metrics to benchmark our endpoints.

• AVG: The average response time in milliseconds
• MED: The median response time in milliseconds
• 90%: 90 percent of all requests were handled within the response time in milliseconds or less.
• 95%: 95 percent of all requests were handled within the response time in milliseconds or less.
• 99%: 99 percent of all requests were handled within the response time in milliseconds or less.
• MIN: The minimum response time in milliseconds
• MAX: The maximum response time in milliseconds
• ERR: The error rate in percent
• R/S: The number of requests per second that could be handled
• MEM: The maximum amount of memory used by the service during the benchmark in MB
• LD: The average system load on the service machine during the benchmark

Update 100.000 products

Metric	Impure	Pure
AVG (ms)	78	12
MED (ms)	75	11
90 (%)	104	16
95 (%)	115	20
99 (%)	140	34
MIN (ms)	42	5
MAX (ms)	798	707
ERR (%)	0%	0%
R/S (sec)	125.66	765.26
MEM (MB)	1176	1279
LD	16	8

Updating existing products results in nearly the same picture as the “create products” benchmark. Interestingly, the impure service performs about 20% better on an update than on a create.

The other metrics are nearly identical to the first benchmark. The pure service uses a bit more memory (around 8%) but is around 6 times faster than the impure one, causing only half of the system load.