Issues filtering by date range using R (httr2)

I’m having trouble when I send an API query like this one:

https://api.quant-aq.com/device-api/v1/devices/<SN>/data/?filter=timestamp_local,ge,2022-11-20;timestamp_local,le,2022-11-27

The response that I get recognizes the first filtering term, but ignores the second one. In other words, I receive data from 2022-11-20 up to today instead of up to 2022-11-27. Is my syntax correct? I’m pretty sure that I’m following the instructions on the API guide.

Thanks!

Hi @mdeighan,

Thanks for posting! I tried to run the same query with one of our internal sensors, and it seems to be returning what I expect. How are you trying to run this query? Are you running it on the command line? With a tool like Postman? If running from the command line, it’s possible that you may need to include quotes around the entire string, as some terminal applications have a hard time escaping strange characters. For example, if using httpie, you could run as:

$ http -a APIKEY: "https://api.quant-aq.com/device-api/v1/devices/<SN>/data/?filter=timestamp_local,ge,2022-11-20;timestamp_local,le,2022-11-27"

Another way to run this same query (in a more performant way) would be to use the bydate endpoint(s) that were introduced earlier this year.

Hi @dhhagan,

Thanks for following up. I’m querying via the httr2 package available in R. The package allows me to preview outgoing queries. Here is what I’m sending (SN and auth credentials redacted):

GET /device-api/v1/devices/<SN>/data/?filter=timestamp_local,ge,2022-11-20;timestamp_local,le,2022-11-27 HTTP/1.1
Host: api.quant-aq.com
User-Agent: httr2/0.2.2 r-curl/4.3.3 libcurl/7.64.1
Accept: */*
Accept-Encoding: deflate, gzip
Authorization: <REDACTED>

I can also run the query with limit and sort options, and these work as expected. The error I’m experiencing is not limited to the date range either. For example, I tried querying PM2.5 values “ge” 0 and “le” 5, and received values well above 5 in the response. I also tested with more than two filter terms. From my end, it looks like only the first filter term is being recognized but any that follow are ignored.

I have successfully used the bydate endpoint, so for now I will use this as a workaround. But may need to use multiple filter terms in the future depending on our analysis. Since it looks like we’re using different software, please let me know if you need more information or if there is anything I can troubleshoot on my end.

Thanks for that additional information. My best guess is that it needs to be enclosed in quotes when sent over the wire - I’m not sure how to do that in httr2, but we can look into it. I can confirm that it’s not an issue when issuing the commands directly with curl or httpie, so it’s not an issue on the API side.

I will dig into httr2 a bit to see if I can figure out what’s going on. Here is a screenshot of the httpie web app displaying results for one of our internal sensors with filter=pm25,gt,10;pm1,lt,1:

Ok, thanks. I’m still a little confused when you say the query needs to be enclosed in quotes. Enclosing in quotes classifies the query as a string. I do pass the query in quotes and as a string to httr2. It’s just that the preview omits printing the quotes for readability. The query would not execute at all if the query were not in quotes. If you’re experimenting with httr2, here is a simple example of a request:

# Prepare request
key <- <API key>
query <- "https://api.quant-aq.com/device-api/v1/<...>"
req <- request(query) %>%
       req_auth_basic(username = key, password = "")

# Preview request before sending (this is what I showed in my post above)
req %>% req_dry_run()

# Perform request
response <- req_perform()

I’m going to ask my IT contact to see whether anything is happening at our firewall. The weird part is that this only happens with filter settings. Like I said, I can add “limit” or “sort” options to the end of the query. It’s weird that filtering terms in the middle of the string are being ignored.