Push Metrics
How to push metrics into Gradient Metrics system
In order to push metrics from your Experiment or Deployment code, you must import gradient-utils from gradient package:

Installing Gradient Utils

1
pip install gradient-utils
Copied!

Instrumenting

Four types of metrics are offered: Counter, Gauge, Summary, and Histogram.

Counter

Counters go up, and reset when the process restarts.
1
from gradient_utils.metrics import Counter
2
c = Counter('my_failures', 'Description of counter')
3
c.inc() # Increment by 1
4
c.inc(1.6) # Increment by given value
Copied!
If there is a suffix of _total on the metric name, it will be removed. When exposing the time series for counter, a _total suffix will be added. This is for compatibility between OpenMetrics and the Prometheus text format, as OpenMetrics requires the _total suffix.
There are utilities to count exceptions raised:
1
@c.count_exceptions()
2
def f():
3
pass
4
5
with c.count_exceptions():
6
pass
7
8
# Count only one type of exception
9
with c.count_exceptions(ValueError):
10
pass
Copied!

Gauge

Gauges can go up and down.
1
from gradient_utils.metrics import Gauge
2
g = Gauge('my_inprogress_requests', 'Description of gauge')
3
g.inc() # Increment by 1
4
g.dec(10) # Decrement by given value
5
g.set(4.2) # Set to a given value
Copied!
There are utilities for common use cases:
1
g.set_to_current_time() # Set to current unixtime
2
3
# Increment when entered, decrement when exited.
4
@g.track_inprogress()
5
def f():
6
pass
7
8
with g.track_inprogress():
9
pass
Copied!
A Gauge can also take its value from a callback:
1
d = Gauge('data_objects', 'Number of objects')
2
my_dict = {}
3
d.set_function(lambda: len(my_dict))
Copied!

Summary

Summaries track the size and number of events.
1
from gradient_utils.metrics import Summary
2
s = Summary('request_latency_seconds', 'Description of summary')
3
s.observe(4.7) # Observe 4.7 (seconds in this case)
Copied!
There are utilities for timing code:
1
@s.time()
2
def f():
3
pass
4
5
with s.time():
6
pass
Copied!
The Python client doesn't store or expose quantile information at this time.

Histogram

Histograms track the size and number of events in buckets. This allows for aggregatable calculation of quantiles.
1
from gradient_utils.metrics import Histogram
2
h = Histogram('request_latency_seconds', 'Description of histogram')
3
h.observe(4.7) # Observe 4.7 (seconds in this case)
Copied!
The default buckets are intended to cover a typical web/rpc request from milliseconds to seconds. They can be overridden by passing buckets keyword argument to Histogram.
There are utilities for timing code:
1
@h.time()
2
def f():
3
pass
4
5
with h.time():
6
pass
Copied!

Labels

All metrics can have labels, allowing grouping of related time series.
Taking a counter as an example:
1
from gradient_utils.metrics import Counter
2
c = Counter('my_requests_total', 'HTTP Failures', ['method', 'endpoint'])
3
c.labels('get', '/').inc()
4
c.labels('post', '/submit').inc()
Copied!
Labels can also be passed as keyword-arguments:
1
from gradient_utils.metrics import Counter
2
c = Counter('my_requests_total', 'HTTP Failures', ['method', 'endpoint'])
3
c.labels(method='get', endpoint='/').inc()
4
c.labels(method='post', endpoint='/submit').inc()
Copied!

Example Code

1
from gradient_utils.metrics import MetricsLogger
2
3
logger = MetricsLogger(grouping_key={'ProjectA': 'SomeLabel'})
4
5
logger.add_gauge("Gauge")
6
logger.add_counter("Counter")
7
8
9
while datetime.now() <= endAt:
10
randNum = randint(1, 100)
11
logger["Gauge"] = 5
12
logger["Gauge"].set(randNum)
13
logger["Counter"].inc()
14
logger.push_metrics()
Copied!
You have to remember to import MetricsLogger:
1
from gradient_utils.metrics import MetricsLogger
Copied!

Notes:

Gradient uses Prometheus behind the scenes. See the Prometheus documentation on metric types and instrumentation best practices the best practices on naming and labels on how to use them.
Last modified 1yr ago