Getting Started With Hosted Graphite

Hosted Graphite is a provider of metric collection, storage and visualisation services. The following diagram gives an overview:

Basic Use Case

Metrics

A metric is a time-series collection of name-value data which is used to populate graphs on your Hosted Graphite dashboard. These metrics and graphs give you the ability to monitor trends, load, reliability, or anything else you can think of!

A Simple Example

A common monitoring target is the number of concurrent users in a web application. A possible solution is to have a scheduled background task, which once a minute queries the appropriate resource and sends us the current number of users. This metric is given an appropriate identifier, for example conc_users. Each time we receive a message with a metric (or metrics), we store the values with the current timestamp (or a timestamp that you can optionally include). When you log into your dashboard composer, you can start graphing this newly available metric with just a few clicks:

Example of 'conc_user' metric graph

Example of ‘conc_user’ metric graph

Simple, but Powerful

Although it lets you get up and running quickly with simple graphs, the dashboard composer is a very powerful tool. You can build complex graphs featuring multiple metrics, with transformations on that metric data - as well as being able to fine tune the styling of the graph.

Here’s another example: this time two metrics are being graphed simultaneously, and some styling has been applied to improve clarity. Check out the Graph Menu Reference for some of the many available possibilities.

Example of 'conc_user' metric

Graphing two related metrics

Metric Data Format

Hosted Graphite metrics should use the following format:

metricname value [timestamp]
  • metricname is a period-delimited path, such as servers.mario.memory.free The periods will turn each path component into a sub-tree. The graphite project website has some metric naming advice.
  • value is an integer or floating point number.
  • timestamp (optional) is a UNIX timestamp, which is the number of seconds since Jan 1st 1970 (always UTC, never local time). If no timestamp is provided, the current time will be assumed. This is probably “good enough” for most uses.

You can send multiple metric values at the same time by putting them on separate lines in the same message:

conc_users 59
free_ram 33

Here’s an example of a metric with the optional UNIX timestamp. This means that the metric will be stored as being measured at that time, instead of the arrival time at our server. This is good for sending us historical data, or when you need the timestamps to match up exactly with the timestamps on your system logs.

conc_users 59 1340981282

Remember that UTC time must be used, not local time. The appropriate time zone adjustments can be applied when you create your graphs.

API Key

Every Hosted Graphite account has an associated API Key. This API Key must be sent along with your metric data to associate it with your account. In most cases this means prefixing the key to your your metric names, e.g. YOUR-API-KEY.metricname, but have a look at the appropriate section below for full instructions.

If the metrics that you send do not appear in your dashboard composer, it usually indicates an incorrect API Key. Refer to our troubleshooting guide.

Sending Metrics to us

There are many ways to send metrics to us. The most basic ways include TCP/UDP messages, HTTP POST, or letting a service/daemon (such as StatsD) perform the work (perhaps after performing some data aggregation).

We also have plugins for popular PAAS services such as Heroku and cloudControl. See our add-ons guide for a full list of platforms and plugins that we support.

Tip: Send your metrics to us in a non-blocking/asynchronous manner. Holding up processing to send metrics will skew the data (if it’s being used for performance analysis), and in the worst case have a negative impact on the responsiveness of your app to users! Below we’ll discuss some simple ways of sending your metric data to us.

TCP Connection

A simple way to send metrics to us is via a TCP connection. First, ensure each metric name is prefixed by your API key (see above), then send your metric(s) using the Metric Data Format.

Host: carbon.hostedgraphite.com Port: 2003

It is not necessary (nor recommended) to create a new connection for each metric. You can put multiple metrics on separate lines. A long-lived connection will be faster because you won’t have to wait to establish a new connection each time. The following simple example shows how to send a single metric using Netcat on linux:

echo "YOUR-API-KEY.conc_users 1.2" | nc carbon.hostedgraphite.com 2003

For programming language-specific examples of sending metrics via TCP, check out our Language Guide.

UDP Packets

For a similar, but non-blocking alternative to TCP, you can use UDP packets to send the metric(s) to us. Ensure each metric name is prefixed by your API key (see above), then send your metric(s) using the Metric Data Format to:

Host: carbon.hostedgraphite.com Port: 2003

Here’s a simple example using netcat on linux:

echo "YOUR-API-KEY.conc_users 59" | nc -uw0 carbon.hostedgraphite.com 2003

For programming language-specific examples of sending metrics via UDP, check out our Language Guide.

HTTP POST

POST your metrics to the following URL:

https://YOUR-API-KEY-HERE@www.hostedgraphite.com/api/v1/sink

Your metric data should be provided in the Metric Data Format as the HTTP POST payload. Don’t prefix your metric names with your API Key (it should already be provided in the URL).

Here’s an example using curl on linux.

curl https://YOUR-API-KEY@www.hostedgraphite.com/api/v1/sink --data-binary "conc_users 59"

If your data was accepted, you will receive a HTTP/1.1 202 Accepted response, with no content body.

For programming language-specific examples of sending metrics via HTTP POST, check out our Language Guide.

TCP Over TLS

If you want you can send your metrics over TCP using an encrypted tunnel. First, ensure each metric name is prefixed by your API key (see above), then send your metric(s) using the Metric Data Format.

Host: carbon.hostedgraphite.com Port: 20030

It is not necessary (nor recommended) to create a new connection for each metric. You can put multiple metrics on separate lines. A long-lived connection will be faster because you won’t have to wait to establish a new connection each time. The following simple example shows how to send a single metric using Ncat on linux:

echo "YOUR-API-KEY.con_users 1.3" | ncat --ssl carbon.hostedgraphite.com 20030

Alternatively, the following example using OpenSSL should work out of the box in OS X:

echo "YOUR-API-KEY.con_users 1.3" |  openssl s_client -connect carbon.hostedgraphite.com:20030

For programming language-specific examples of sending metrics via TLS over TCP, check out our Language Guide.

Quick Reference

You can send Graphite messages via:

  • TCP & UDP connections to carbon.hostedgraphite.com port 2003 (Metric names must be API Key-prefixed)
  • TCP over TLS tunnel to carbon.hostedgraphite.com port 20030 (Metric names must be API Key-prefixed)
  • HTTP POST to https://YOUR-API-KEY@www.hostedgraphite.com/api/v1/sink (Metric names are not API Key-prefixed, HTTP 202 response on success)
  • Python Pickle carbon.hostedgraphite.com port 2004 (discussed in the language guide)