Unable to retrieve sensor data through API for over 2 months.

I have not been able to retrieve external sensors data through the API (runtimeReport/includeRuntimeSensorReport) since mid-June. It does work some times but mostly not.

Any status that you can share about this ongoing problem?
I know you're upgrading your infrastructure but beyond that?
1 person has
this problem
+1
Reply
  • Hi,

    Can you provide a few more details for us to look into this further? What exactly is the request payload you are sending and what is the full HTTP response you get when you say it does not work? Is it a timeout, or empty response, etc.?

    Also we don't have a includeRuntimeSensorReport parameter, so are you referring to the includeSensor boolean that is part of the thermostat selector that you can pass to runtimeReport/ or GET thermostat/?

    We are working on our infrastructure capacity as we are currently experiencing periods where runtime and sensor data gets periodically dropped throughout the day. Perhaps these dropped windows are corresponding to when you are not getting data from the API?

    We are actively working on investing a lot more in our infrastructure. Unfortunately this is not a very fast process as it involves many facets from our own software changes, to middleware, to physical data center additions. It will be a progressive set of changes over the next few months but the loss of runtime data throughout the day should be resolved in the next two weeks. Thanks for your patience.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. happy, confident, thankful, excited kidding, amused, unsure, silly indifferent, undecided, unconcerned sad, anxious, confused, frustrated

  • Hesham,

    Here is a request for the last 3 updates:

    URL: https://api.ecobee.com/1/runtimeRepor...
    Response body: {
    "reportList": [],
    "sensorList": [
    {
    "thermostatIdentifier": "xxx",
    "sensors": [],
    "columns": [],
    "data": []
    }
    ],
    "startDate": "2015-09-04",
    "startInterval": 189,
    "endDate": "2015-09-04",
    "endInterval": 191,
    "status": {
    "code": 0,
    "message": ""
    }
    }

    If I request all the data since the beginning of the day (not just the last 3 updates):

    URL: https://api.ecobee.com/1/runtimeRepor...
    Response body: {
    "reportList": [],
    "sensorList": [
    {
    "thermostatIdentifier": "xxx",
    "sensors": [
    {
    "sensorId": "ei:0:1",
    "sensorName": "Water Heater",
    "sensorType": "dryContact",
    "sensorUsage": "monitor"
    },
    {
    "sensorId": "ei:0:2",
    "sensorName": "Water Pump",
    "sensorType": "dryContact",
    "sensorUsage": "monitor"
    },
    {
    "sensorId": "rs:0:1",
    "sensorName": "Upstairs",
    "sensorType": "temperature",
    "sensorUsage": "indoor"
    },
    {
    "sensorId": "rs:0:3",
    "sensorName": "Furnace In",
    "sensorType": "temperature",
    "sensorUsage": "monitor"
    },
    {
    "sensorId": "rs:0:4",
    "sensorName": "Furnace Out",
    "sensorType": "temperature",
    "sensorUsage": "monitor"
    },
    {
    "sensorId": "rs:0:2",
    "sensorName": "Outside",
    "sensorType": "temperature",
    "sensorUsage": "outdoor"
    }
    ],
    "columns": [
    "date",
    "time",
    "ei:0:1",
    "ei:0:2",
    "rs:0:1",
    "rs:0:3",
    "rs:0:4",
    "rs:0:2"
    ],
    "data": [
    "2015-09-03,20:00:00,1,1,79.5,77.8,69.9,72.2",
    "2015-09-03,20:05:00,1,1,79.7,76.6,68.9,71.8",
    "2015-09-03,20:10:00,1,1,79.7,76.2,69.2,71.5",
    "2015-09-03,20:15:00,null,null,,,,",
    "2015-09-03,20:20:00,null,null,,,,",
    "2015-09-03,20:25:00,null,null,,,,",
    "2015-09-03,20:30:00,null,null,,,,",
    "2015-09-03,20:35:00,null,null,,,,",
    "2015-09-03,20:40:00,null,null,,,,",
    "2015-09-03,20:45:00,1,1,79.6,75.8,73.2,70.3",
    "2015-09-03,20:50:00,1,1,79.7,75.8,74.3,70.1",
    "2015-09-03,20:55:00,1,1,79.6,75.7,66.6,70.2",
    "2015-09-03,21:00:00,1,1,79.6,75,56.9,69.7",
    "2015-09-03,21:05:00,1,1,79.4,74.5,55.7,69",
    "2015-09-03,21:10:00,1,1,79.3,74,55,68.9",
    "2015-09-03,21:15:00,null,null,,,,",
    "2015-09-03,21:20:00,null,null,,,,",
    "2015-09-03,21:25:00,null,null,,,,",
    "2015-09-03,21:30:00,1,1,79,75.8,65.6,68.6",
    {data is present between 21:35 and 11:05}
    "2015-09-04,11:10:00,1,1,75.7,73.9,73.5,68.3",
    "2015-09-04,11:15:00,null,null,,,,",
    "2015-09-04,11:20:00,null,null,,,,",
    "2015-09-04,11:25:00,null,null,,,,",
    "2015-09-04,11:30:00,null,null,,,,",
    "2015-09-04,11:35:00,null,null,,,,",
    "2015-09-04,11:40:00,null,null,,,,",
    "2015-09-04,11:45:00,null,null,,,,",
    "2015-09-04,11:50:00,null,null,,,,",
    "2015-09-04,11:55:00,null,null,,,,",
    "2015-09-04,12:00:00,null,null,,,,",
    "2015-09-04,12:05:00,null,null,,,,",
    "2015-09-04,12:10:00,null,null,,,,"
    ]
    }
    ],
    "startDate": "2015-09-04",
    "startInterval": 0,
    "endDate": "2015-09-04",
    "endInterval": 194,
    "status": {
    "code": 0,
    "message": ""
    }
    }

    This is what I mean.
    Good luck with your current infrastructure upgrade.

    Thank you,
    Daniel Born
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. happy, confident, thankful, excited kidding, amused, unsure, silly indifferent, undecided, unconcerned sad, anxious, confused, frustrated

  • According to a recently received email from ecobee, all issues regarding homeiq data should be resolved.

    However, I am still seeing issues with external sensors as mentioned above. Is this something that still needs to be addresses or is this a different situation?
    Last few times I checked, it seems like all data is there except for the last hour's worth (like the data makes it there but with an hour delay).

    Please advise,
    Daniel Born
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. happy, confident, thankful, excited kidding, amused, unsure, silly indifferent, undecided, unconcerned sad, anxious, confused, frustrated

  • Hi Daniel,
    Yes, as per the recent communication all of the issues have been resolved.
    You are correct though, there is an hour latency and this is the expected behaviour (in fact there may be a further communication from ecobee regarding this in the coming days).
    Note: there is no missing data, and as per the communication those issues have been resolved - what you are observing is the 1 hour latency which is there by design and not by accident.

    I hope this helps,
    Scott
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. sad, anxious, confused, frustrated kidding, amused, unsure, silly indifferent, undecided, unconcerned happy, confident, thankful, excited

  • So you are saying that while in the past it was possible to get data about 15 minutes old or less, now the new norm will be at least one hour old?

    So this will be normal? (data requested at 14:45):
    "2015-09-15,12:20:00,1,1,74.4,75.2,74.1,70.1",
    "2015-09-15,12:25:00,1,1,74.5,75.2,74.1,70.8",
    "2015-09-15,12:30:00,null,null,74.4,75.2,74.1,70.6",
    "2015-09-15,12:35:00,null,null,74.6,75.2,74.1,71",
    "2015-09-15,12:40:00,null,null,74.5,75.2,74.1,71",
    "2015-09-15,12:45:00,null,null,74.5,75.2,74.1,71",
    "2015-09-15,12:50:00,null,null,74.6,75.3,74.1,71",
    "2015-09-15,12:55:00,null,null,74.7,75.3,74.1,71.7",
    "2015-09-15,13:00:00,null,null,74.6,75.3,74.2,71.6",
    "2015-09-15,13:05:00,null,null,74.6,75.3,74.2,71.9",
    "2015-09-15,13:10:00,null,null,74.7,75.3,74.2,72",
    "2015-09-15,13:15:00,null,null,74.8,75.3,74.2,72.1",
    "2015-09-15,13:20:00,null,null,74.8,75.4,74.2,72.1",
    "2015-09-15,13:25:00,null,null,74.8,75.4,74.3,72.7",
    "2015-09-15,13:30:00,null,null,74.8,75.4,74.3,73",
    "2015-09-15,13:35:00,null,null,74.9,75.5,74.3,72.8",
    "2015-09-15,13:40:00,null,null,75,75.5,74.3,72.9",
    "2015-09-15,13:45:00,null,null,,,,",
    "2015-09-15,13:50:00,null,null,,,,",
    "2015-09-15,13:55:00,null,null,,,,",
    "2015-09-15,14:00:00,null,null,,,,",
    "2015-09-15,14:05:00,null,null,,,,",
    "2015-09-15,14:10:00,null,null,,,,",
    "2015-09-15,14:15:00,null,null,,,,",
    "2015-09-15,14:20:00,null,null,,,,",
    "2015-09-15,14:25:00,null,null,,,,"

    This is going to be a problem.
    • Hi Daniel,

      Yes, the runtimeReport will always be delayed by approximately 1hr. The runtimeReport data comes from our data warehouse, which was not guaranteed to be realtime. This is because there are various back end processes involved in rolling up the data and making it available for querying in any given interval range.

      If what you are looking for is getting the last 15 minute intervals then the real-time mechanism for that is obtaining the ExtendedRuntime on the Thermostat (https://www.ecobee.com/home/developer...) as this will always have the last 15 minute interval reported by the thermostat. For the sensors the RemoteSensorCapability (https://www.ecobee.com/home/developer...) will have the latest readings from each of the sensors in real time.

      Hope this helps.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. happy, confident, thankful, excited kidding, amused, unsure, silly indifferent, undecided, unconcerned sad, anxious, confused, frustrated

  • 1
    Hi Daniel,

    Sorry for the delay in responding. I was meaning to reply sooner.

    The issue you originally reported where two different requests that overlap the same interval range show different results is the outcome of two different issues. The first issue is that the API does not pad the results with null values if there is no data in the interval requested. So in your first request (start interval 189, end interval 191) there was no data available, so the API returned an empty response. However for the second request (start interval 0, end interval 191) there was data mid way through this interval, so the API response pads the remaining segments of the interval with nulls.

    As you know, the reason why you were having no data was due to the HomeIQ data gaps we were experiencing. We have now modified parts of our back end processing (both infrastructure and middleware) and these data gaps are now resolved.

    We are currently experiencing an API only bug where a runtimeReport request that has a start date of the current date will always return empty data. This was unfortunately introduced as a result of our recent changes. We are working on a quick remedy to this and should have it resolved by next week. In the meantime, a workaround is to have the runtimeReport requests always start from prior to the current date.

    Thanks again for your patience.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. sad, anxious, confused, frustrated kidding, amused, unsure, silly indifferent, undecided, unconcerned happy, confident, thankful, excited

  • Hi,

    I have the same issue on my end. My requests ask for the dehumifier runtime in the last hour.

    For the moment, the only response I get from your report runtime APIs is the following:

    doRequest>exception groovyx.net.http.HttpResponseException: Internal Server Error for null

    This used to work before the HomeIQ data mess.

    Regards.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. sad, anxious, confused, frustrated kidding, amused, unsure, silly indifferent, undecided, unconcerned happy, confident, thankful, excited

  • Yves,

    We have not seen the issue you are reporting, and all ecobee's own interfaces (e.g. web portal and mobile apps) use the same APIs. Can you please confirm the exact request payloads and the actual HTTP responses to confirm that the requests are not invalid?
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. sad, anxious, confused, frustrated kidding, amused, unsure, silly indifferent, undecided, unconcerned happy, confident, thankful, excited

  • Hello @Hesham,

    In fact, after checking further, I can see that my requests now are executed properly, but I receive no values.

    Here is my request:

    https://api.ecobee.com/1/runtimeRepor...=
    {
    "startInterval":"140",
    "endInterval":"152",
    "startDate":"2015-09-15",
    "endDate":"2015-09-15",
    "columns":"dehumidifier",
    "includeSensors":"false",
    "selection":{
    "selectionType":"thermostats",
    "selectionMatch":"XXXXXXXXXXXX"
    }
    }

    The answer from the API is the following:

    reportList= [[rowList:[2015-09-15,07:35:00,,, 2015-09-15,07:40:00,,, 2015-09-15,07:45:00,,, 2015-09-15,07:50:00,,, 2015-09-15,07:55:00,,, 2015-09-15,08:00:00,,, 2015-09-15,08:05:00,,, 2015-09-15,08:10:00,,, 2015-09-15,08:15:00,,, 2015-09-15,08:20:00,,, 2015-09-15,08:25:00,,, 2015-09-15,08:30:00,,], rowCount:12, thermostatIdentifier:XXXXXXXXXXX]]

    Based on your answer provided above, it looks like now we can expect an hour latency when we query the Runtime report APIs.

    What should I do now? Should I query the extended Runtime Object?

    https://www.ecobee.com/home/developer...

    The way I understand it though is that the extendedRuntime object only
    contains the last 15 minutes of runtime data. So, there is a big data gap.

    I used to get the runtime report data in order to control my dehumidfier/HRV and turn it on/off for a minimum runtime in an hour (sort of a ventilatorMinTime which is not currently implemented by ecobee yet).

    Let me know. Thnx.

    Regards.
    • view 3 more comments
    • @Hesham, I'm already getting the runtimeRevision from thermostatSummary. Is the hour latency a new rule or it's been there before? I've never heard of it before. Is there a reference in the ecobee documentation?

      Thnx again for your support!
    • @yracine, sorry for replying too fast, not fully understanding your question. I agree, if the runtime data is 1 hour old, the extended runtime should be able to return 1 hour's worth of data and not just the last 15 minutes.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. happy, confident, thankful, excited kidding, amused, unsure, silly indifferent, undecided, unconcerned sad, anxious, confused, frustrated

  • I don't think these remoteSensor and remoteSensorCapability objects existed at the time I started using the runtimeReports to access the remote sensor data.

    This is even better for me now as all I wanted was the most up-to-date remote sensor data.

    Many thanks,
    Daniel
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. happy, confident, thankful, excited kidding, amused, unsure, silly indifferent, undecided, unconcerned sad, anxious, confused, frustrated