- If you’ve never used Graphite before, take a look at the Getting Started guide to get a quick overview of how to start sending metrics.
- If you’re familiar with Graphite, or want to know how our version works, read on to find out how Hosted Graphite differs from standard Graphite.
- Check out our Roadmap for a look at upcoming features and services
Hosted Graphite is a collection of services which work together to help you collect, store and visualise metric data from any source. Most commonly metrics are gathered through monitoring programs like Diamond, CollectD, or our own Hosted Graphite Agent.
The following diagram gives an overview:
There are a lot of differences between HG Graphite and standard Graphite, but there are 2 which will affect users the most.
- Data aggregation is automatic. In standard Graphite only one datapoint is saved per resolution interval, so if you want to send data more frequently you need to use some kind of pre-aggregation service, like StatsD or Carbon Aggregator. Hosted Graphite creates Data Views for all metrics we receive, which consolidate all your datapoints like an aggregator would, but also gives you access to 9+ different versions of your aggregated metrics. Visit the Data Views page for a full list of all the data views you can access.
- No ‘Last Write Wins’. As mentioned above, in normal Graphite only one datapoint per timestamp can be stored. If a new datapoint is sent to the old timestamp, it overwrites the old value. With Hosted Graphite the new datapoint is incorporated into the data views instead.
A metric is a time-series collection of name-value data which is used to populate graphs on your Hosted Graphite dashboard. These metrics and graphs give you the ability to monitor trends, load, reliability, or anything else you can think of!
A common monitoring target is the number of concurrent users in a web application. A possible solution is to have a scheduled background task, which once a minute queries the appropriate resource and sends us the current number of users. This metric is given an appropriate identifier, for example conc_users. Each time we receive a message with a metric (or metrics), we store the values with the current timestamp (or a timestamp that you can optionally include). When you create a new Grafana dashboard, you can start graphing this newly available metric with just a few clicks:
Although it lets you get up and running quickly with simple graphs, Grafana is a very powerful tool. You can build complex graphs featuring multiple metrics, with transformations on that metric data - as well as being able to fine tune the styling of the graph.
Here’s another example: this time three metrics are being graphed simultaneously, and some styling has been applied to improve clarity. Check out the Grafana graphs panel reference for some of the many available possibilities.
Hosted Graphite metrics should use the following format:
metricname value [timestamp]
- metricname is a period-delimited path, such as servers.mario.memory.free The periods will turn each path component into a sub-tree. The graphite project website has some metric naming advice.
- value is an integer or floating point number.
- timestamp (optional) is a UNIX timestamp, which is the number of seconds since Jan 1st 1970 (always UTC, never local time). If no timestamp is provided, the current time will be assumed. This is probably “good enough” for most uses.
You can send multiple metric values at the same time by putting them on separate lines in the same message:
conc_users 59 free_ram 33
Here’s an example of a metric with the optional UNIX timestamp. This means that the metric will be stored as being measured at that time, instead of the arrival time at our server. This is good for sending us historical data, or when you need the timestamps to match up exactly with the timestamps on your system logs.
conc_users 59 1340981282
Remember that UTC time must be used, not local time. The appropriate time zone adjustments can be applied when you create your graphs.
Every Hosted Graphite account has an associated API Key. This API Key must be sent along with your metric data to associate it with your account. In most cases this means prefixing the key to your your metric names, e.g. YOUR-API-KEY.metricname, but have a look at the appropriate section below for full instructions.
There are many ways to send metrics to us. The most basic ways include TCP/UDP messages, HTTP POST, or letting a service/daemon (such as StatsD) perform the work (perhaps after performing some data aggregation).
Tip: Send your metrics to us in a non-blocking/asynchronous manner. Holding up processing to send metrics will skew the data (if it’s being used for performance analysis), and in the worst case have a negative impact on the responsiveness of your app to users! Below we’ll discuss some simple ways of sending your metric data to us.
Host: carbon.hostedgraphite.com Port: 2003
It is not necessary (nor recommended) to create a new connection for each metric. You can put multiple metrics on separate lines. A long-lived connection will be faster because you won’t have to wait to establish a new connection each time. The following simple example shows how to send a single metric using Netcat on linux:
echo "YOUR-API-KEY.conc_users 1.2" | nc carbon.hostedgraphite.com 2003
For programming language-specific examples of sending metrics via TCP, check out our Language Guide.
For a similar, but non-blocking alternative to TCP, you can use UDP packets to send the metric(s) to us. Ensure each metric name is prefixed by your API key (see above), then send your metric(s) using the Metric Data Format to:
Host: carbon.hostedgraphite.com Port: 2003
Here’s a simple example using netcat on linux:
echo "YOUR-API-KEY.conc_users 59" | nc -uw0 carbon.hostedgraphite.com 2003
You can send multiple metrics on separate lines as with TCP, but be aware there is a size limit of 8192 bytes for UDP packets.
For programming language-specific examples of sending metrics via UDP, check out our Language Guide.
POST your metrics to the following URL:
Here’s an example using curl on linux.
curl https://YOUR-API-KEY@www.hostedgraphite.com/api/v1/sink --data-binary "conc_users 59"
If your data was accepted, you will receive a HTTP/1.1 202 Accepted response, with no content body.
For programming language-specific examples of sending metrics via HTTP POST, check out our Language Guide.
If you want you can send your metrics over TCP using an encrypted tunnel. First, ensure each metric name is prefixed by your API key (see above), then send your metric(s) using the Metric Data Format.
Host: carbon.hostedgraphite.com Port: 20030
It is not necessary (nor recommended) to create a new connection for each metric. You can put multiple metrics on separate lines. A long-lived connection will be faster because you won’t have to wait to establish a new connection each time. The following simple example shows how to send a single metric using Ncat on linux:
echo "YOUR-API-KEY.con_users 1.3" | ncat --ssl carbon.hostedgraphite.com 20030
Alternatively, the following example using OpenSSL should work out of the box in OS X:
echo "YOUR-API-KEY.con_users 1.3" | openssl s_client -connect carbon.hostedgraphite.com:20030
For programming language-specific examples of sending metrics via TLS over TCP, check out our Language Guide.
You can send Graphite messages via:
- TCP & UDP connections to carbon.hostedgraphite.com port 2003 (Metric names must be API Key-prefixed)
- TCP over TLS tunnel to carbon.hostedgraphite.com port 20030 (Metric names must be API Key-prefixed)
- HTTP POST to https://YOUR-API-KEY@www.hostedgraphite.com/api/v1/sink (Metric names are not API Key-prefixed, HTTP 202 response on success)
- Python Pickle carbon.hostedgraphite.com port 2004 (discussed in the language guide)