Using Coinmarketcap Pro and CryptoCompare APIs around manipulating timestamps in Python

You’re building your backend web services with Python and things are going swimmingly. Apart from one thing — every service seems to be using a different timestamp format. Whilst we wish that all services would stick to one format, this is just not the case.

Luckily Python provide the tools to convert timestamps of various formats into a datetime object, where we can then manipulate them as we wish. This article gives you a taster of how to do this.

Checking your system timezone

Before we manipulate time in Python, make sure you know which timezone your system is running:

import timeprint(time.tzname)
print(time.time())> [UTC, UTC]

time.time() gives us 2 values, the timezone not accounting for daylight savings, and the timezone accounting for daylight savings. What if we don’t want to worry about DST issues? Stick to UTC time.

The switch to daylight saving time does not affect UTC. It refers to time on the zero or Greenwich meridian, which is not adjusted to reflect changes either to or from Daylight Saving Time. — nhc.noaa.gov

This is why UTC stands for Universal Time Coordinated, it makes our lives as programmers easier to use a universally accepted format.

Use the following command if you need to switch your timezone to UTC:

sudo timedatectl set-timezone UTC

Converting a timestamp from a Coinmarketcap Pro API call

Let’s move onto a real world example of fetching an API response where a timestamp is a necessity to store or manipulate. In this case let’s use the new Coinmarketcap Pro API to fetch the latest Bitcoin prices.

Note: Both services used in this article adhere to UTC time by default.

To make the call let’s use Python’s requests module, and format the result as a JSON Object:
(Install requests with pip install requests)

import time, requests, json
import datetime as dturl = 'https://pro-api.coinmarketcap.com/v1/cryptocurrency/quotes/latest?symbol=BTC&convert=USD'headers = {
 'Accept': 'application/json',
 'Accept-Encoding': 'deflate, gzip',
 'X-CMC_PRO_API_KEY': COINMARKETCAP_PRO_API_KEY,
}r = requests.get(url, headers=headers)if r.status_code == 200:
   response = json.loads(r.text)

The above example is everything you need to start using the Coinmarketcap Pro API with Python. Remember the X-CMC_PRO_API_KEY header, being your API key.

Let’s run print(response) to see what type of timestamp we are dealing with here:

{'status': 
   {
     'timestamp': '2018-08-09T07:26:22.078Z',
     ...
    
    'data': {
      'BTC': {
         'date_added': '2013-04-28T00:00:00.000Z', 
         'last_updated': '2018-08-09T07:25:31.000Z'
          ...
       }
     }
   }
}

Great, it appears that Coinmarketcap have opted to use a verbose yet readable timestamp format in their pro service. The format we need to make note of is year-month-dayThour:minute:seconds.millisecondsZ.

It is important we adhere to this format for datetime to recognise this string to be representitive of a date and time. That means any miscellaneous characters, including the T and Z, need to be included in our datetime formatting.

The Python format is %Y-%m-%dT%H:%M:%S.%fZ. %f will handle the millisecond value of the timestamp. Whether a 2 or 3 digit millisecond value is provided does not matter, Python will still recognise the format we are assigning.

We can continue to build our script to store our retrieved timestamp as a datetime object:

...lastUpdated = response['data']['BTC']['quote']['USD']['last_updated']timestamp = dt.datetime.strptime(lastUpdated, '%Y-%m-%dT%H:%M:%S.000Z')print(timestamp)> 2018-08-09 07:25:31

Note: Coinmarketcap have departed from the unix based timestamps they opted for in the legacy free service (of which is discontinuing in December 2018 — use the free pro API to get started with their services).

Converting timestamps to other formats

Once a datetime object has been initialised we can then format and manipulate it in any way we see fit.

Probably the most useful conversion is that of the unix timestamp. It is standard to store date time data either as a unix timestamp int, or a datetime object directly in databases that support them.

Continuing with the above example, the unix timestamp can be generated using the %s (second) formatter, getting the total seconds passed since 1st Jan 1970.

lastUpdatedUnix = int(lastUpdatedDt.strftime("%s"))

Fantastic, this is now suitable to persist in a database document.

Check out all the formatting options at https://docs.python.org/3/library/datetime.html#strftime-and-strptime-behavior

What if we only wanted the date and not the leading time? We could do:

dateOnly = str(lastUpdatedDt.strftime("%Y-%m-%d"))print(dateOnly)> 2018-08-09dateOnlyUnix = dt.datetime.strptime(lastUpdated, "%s");

Here we are generating a new date string and applying it to another datetime object.

Converting a timestamp from a CryptoCompare API call

Let’s look at another real world example. Let’s get yesterday’s open, high, low and close prices of Bitcoin:

request = requests.get('https://min-api.cryptocompare.com/data/histoday?fsym=BTC&tsym=USD')if request.content != '':
   response = request.content
   response = json.loads(response)
   print(response)

By default this CryptoCompare API call returns an array of daily price records for the past month. A time record is nested within each daily record:

{ 
   'Response': 'Success', 
    ...
    'Data': [
     {
        'time': 1531180800
        ...
     },
     ...
    ]
}

Unlike Coinmarketcap Pro, CryptoCompare have opted for a unix based timestamp.

I want to get the index of yesterday’s record — the last full day the containing price records are based on. Once calculated, we can retreive the data:

dayIndex = len(response['Data']) - 2
  
time       = int(response['Data'][dayIndex]['time'])
priceHigh  = float(response['Data'][dayIndex]['high'])
priceLow   = float(response['Data'][dayIndex]['low'])
priceOpen  = float(response['Data'][dayIndex]['open'])
priceClose = float(response['Data'][dayIndex]['close'])
priceTime  = float(response['Data'][dayIndex]['time'])

Just as before, we can now plug our unix timestamp into a datetime object and format it as we wish. Datetime contains a utcfromtimestamp function that will do exactly this:

timestampDateOnly = dt.datetime.utcfromtimestamp(time).strftime("%d/%m/%Y")print(timestampDateOnly)> 08/08/2018

Adding and Subtracting time using Timedelta

Sometimes we want to generate another date relative to a timestamp, such as tomorrows date, or yesterdays date, perhaps if my supervisor is doing a range of chores based on yesterdays tasks. timedelta can be used for this:

...
from datetime import timedeltatoday = 1533781210tomorrow = dt.datetime.utcfromtimestamp(today) + timedelta(days=1)tomorrow = tomorrow.strftime("%d/%m/%Y")

Here we are using a unix timestamp and datetime.utcfromtimestamp to get the UTC time directly, before adding 1 day to it.

We are not limited to just using days with timedelta. Other supported time intervals are seconds, microseconds, milliseconds, minutes, hours and weeks.

You can also use negative values, E.g. days=-1, if this is more suited to your syntax for subtracting time.

Where to go from here

Both CryptoCompare and Coinmarketcap Pro offer free API services for you to play with.

Using real world services as a means to build utilities in your libraries is a solid strategy for expanding your codebase.