tag:status.faros.ai,2005:/historyFaros AI Status - Incident History2024-03-28T22:23:13-07:00Faros AItag:status.faros.ai,2005:Incident/203527612024-03-27T18:00:02-07:002024-03-27T18:00:02-07:00Planned Maintenance<p><small>Mar <var data-var='date'>27</var>, <var data-var='time'>18:00</var> PDT</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Mar <var data-var='date'>27</var>, <var data-var='time'>17:00</var> PDT</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Mar <var data-var='date'>25</var>, <var data-var='time'>10:17</var> PDT</small><br><strong>Scheduled</strong> - We will be undergoing scheduled maintenance during this time.</p>tag:status.faros.ai,2005:Incident/203310352024-03-25T18:00:02-07:002024-03-25T18:00:02-07:00Planned Maintenance<p><small>Mar <var data-var='date'>25</var>, <var data-var='time'>18:00</var> PDT</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Mar <var data-var='date'>25</var>, <var data-var='time'>17:00</var> PDT</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Mar <var data-var='date'>22</var>, <var data-var='time'>11:36</var> PDT</small><br><strong>Scheduled</strong> - We will be undergoing scheduled maintenance during this time.</p>tag:status.faros.ai,2005:Incident/202743882024-03-18T18:00:56-07:002024-03-18T18:00:56-07:00Planned Maintenance<p><small>Mar <var data-var='date'>18</var>, <var data-var='time'>18:00</var> PDT</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Mar <var data-var='date'>18</var>, <var data-var='time'>17:00</var> PDT</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Mar <var data-var='date'>15</var>, <var data-var='time'>18:29</var> PDT</small><br><strong>Scheduled</strong> - We will be undergoing scheduled maintenance during this time.</p>tag:status.faros.ai,2005:Incident/199793122024-02-14T08:37:18-08:002024-02-14T08:37:49-08:00Elevated response times on GraphQL API<p><small>Feb <var data-var='date'>14</var>, <var data-var='time'>08:37</var> PST</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Feb <var data-var='date'>13</var>, <var data-var='time'>21:29</var> PST</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Feb <var data-var='date'>13</var>, <var data-var='time'>21:06</var> PST</small><br><strong>Identified</strong> - The issue has been identified and a fix is being implemented.</p><p><small>Feb <var data-var='date'>13</var>, <var data-var='time'>14:58</var> PST</small><br><strong>Investigating</strong> - We are currently investigating this issue.</p>tag:status.faros.ai,2005:Incident/199243742024-02-12T17:47:22-08:002024-02-12T17:47:22-08:00Planned Maintenance<p><small>Feb <var data-var='date'>12</var>, <var data-var='time'>17:47</var> PST</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Feb <var data-var='date'>12</var>, <var data-var='time'>17:00</var> PST</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Feb <var data-var='date'> 6</var>, <var data-var='time'>11:13</var> PST</small><br><strong>Scheduled</strong> - We will be undergoing scheduled maintenance during this time.</p>tag:status.faros.ai,2005:Incident/198424702024-01-26T17:46:33-08:002024-01-26T17:46:33-08:00Unplanned Maintenance<p><small>Jan <var data-var='date'>26</var>, <var data-var='time'>17:46</var> PST</small><br><strong>Completed</strong> - Database upgrade completed.</p><p><small>Jan <var data-var='date'>26</var>, <var data-var='time'>17:45</var> PST</small><br><strong>Verifying</strong> - Verification is currently underway for the maintenance items.</p><p><small>Jan <var data-var='date'>26</var>, <var data-var='time'>17:45</var> PST</small><br><strong>Update</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Jan <var data-var='date'>26</var>, <var data-var='time'>17:45</var> PST</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Jan <var data-var='date'>26</var>, <var data-var='time'>17:41</var> PST</small><br><strong>Scheduled</strong> - We will be upgrading the databases serving our dashboards to improve query performance. We are expecting a short downtime dashboards in serving.</p>tag:status.faros.ai,2005:Incident/193801082023-12-11T18:01:27-08:002023-12-11T18:01:27-08:00Planned Maintenance<p><small>Dec <var data-var='date'>11</var>, <var data-var='time'>18:01</var> PST</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Dec <var data-var='date'>11</var>, <var data-var='time'>17:02</var> PST</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Dec <var data-var='date'> 8</var>, <var data-var='time'>12:13</var> PST</small><br><strong>Scheduled</strong> - We will be undergoing scheduled maintenance during this time.</p>tag:status.faros.ai,2005:Incident/193554792023-12-07T19:00:33-08:002023-12-07T19:00:34-08:00Planned Maintenance<p><small>Dec <var data-var='date'> 7</var>, <var data-var='time'>19:00</var> PST</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Dec <var data-var='date'> 7</var>, <var data-var='time'>18:00</var> PST</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Dec <var data-var='date'> 6</var>, <var data-var='time'>10:12</var> PST</small><br><strong>Scheduled</strong> - We will be undergoing scheduled maintenance during this time.</p>tag:status.faros.ai,2005:Incident/192235432023-11-30T18:00:12-08:002023-11-30T18:00:12-08:00Planned Maintenance<p><small>Nov <var data-var='date'>30</var>, <var data-var='time'>18:00</var> PST</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Nov <var data-var='date'>30</var>, <var data-var='time'>17:00</var> PST</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Nov <var data-var='date'>26</var>, <var data-var='time'>20:22</var> PST</small><br><strong>Scheduled</strong> - We will be undergoing scheduled maintenance during this time.</p>tag:status.faros.ai,2005:Incident/191284622023-11-15T08:22:46-08:002023-11-15T08:22:46-08:00Website not loading<p><small>Nov <var data-var='date'>15</var>, <var data-var='time'>08:22</var> PST</small><br><strong>Resolved</strong> - The issue was resolved. We will update the resolution with more details once we collect them.</p><p><small>Nov <var data-var='date'>15</var>, <var data-var='time'>06:10</var> PST</small><br><strong>Monitoring</strong> - We rolled back the previous version of the website and monitoring the operation. <br /><br />We are looking to why the pre-rollout checks did not detect the issue, as well as why the periodic UX checks did not detect the it on time.</p><p><small>Nov <var data-var='date'>15</var>, <var data-var='time'>06:08</var> PST</small><br><strong>Identified</strong> - Due to a invalid version rollout the website returned 500 errors. Rolling back.</p><p><small>Nov <var data-var='date'>15</var>, <var data-var='time'>06:01</var> PST</small><br><strong>Investigating</strong> - We are currently investigating this issue.</p>tag:status.faros.ai,2005:Incident/187813442023-10-23T14:42:52-07:002023-10-23T14:42:52-07:00Web Application intermittently unreachable<p><small>Oct <var data-var='date'>23</var>, <var data-var='time'>14:42</var> PDT</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Oct <var data-var='date'>23</var>, <var data-var='time'>14:42</var> PDT</small><br><strong>Update</strong> - We are continuing to monitor for any further issues.</p><p><small>Oct <var data-var='date'>13</var>, <var data-var='time'>19:06</var> PDT</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Oct <var data-var='date'>13</var>, <var data-var='time'>12:01</var> PDT</small><br><strong>Identified</strong> - The issue has been identified and a fix is being implemented.</p><p><small>Oct <var data-var='date'>13</var>, <var data-var='time'>09:16</var> PDT</small><br><strong>Investigating</strong> - We are currently investigating this issue.</p>tag:status.faros.ai,2005:Incident/187663762023-10-13T18:00:18-07:002023-10-13T18:00:18-07:00Planned Maintenance<p><small>Oct <var data-var='date'>13</var>, <var data-var='time'>18:00</var> PDT</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Oct <var data-var='date'>13</var>, <var data-var='time'>17:00</var> PDT</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Oct <var data-var='date'>12</var>, <var data-var='time'>08:05</var> PDT</small><br><strong>Scheduled</strong> - We will be undergoing scheduled maintenance during this time.</p>tag:status.faros.ai,2005:Incident/187462312023-10-12T07:01:26-07:002023-10-12T07:01:26-07:00Planned Maintenance<p><small>Oct <var data-var='date'>12</var>, <var data-var='time'>07:01</var> PDT</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Oct <var data-var='date'>12</var>, <var data-var='time'>06:01</var> PDT</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Oct <var data-var='date'>10</var>, <var data-var='time'>11:24</var> PDT</small><br><strong>Scheduled</strong> - We will be undergoing scheduled maintenance during this time.</p>tag:status.faros.ai,2005:Incident/186906832023-10-07T09:00:48-07:002023-10-07T09:00:48-07:00Planned Maintenance<p><small>Oct <var data-var='date'> 7</var>, <var data-var='time'>09:00</var> PDT</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Oct <var data-var='date'> 7</var>, <var data-var='time'>08:00</var> PDT</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Oct <var data-var='date'> 4</var>, <var data-var='time'>10:31</var> PDT</small><br><strong>Scheduled</strong> - We will be undergoing scheduled maintenance during this time.</p>tag:status.faros.ai,2005:Incident/186893852023-10-04T10:17:38-07:002023-10-04T10:17:38-07:00Web application not loading<p><small>Oct <var data-var='date'> 4</var>, <var data-var='time'>10:17</var> PDT</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Oct <var data-var='date'> 4</var>, <var data-var='time'>08:34</var> PDT</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Oct <var data-var='date'> 4</var>, <var data-var='time'>08:17</var> PDT</small><br><strong>Identified</strong> - We identified the issue with our auth provider - https://status.frontegg.com/incidents/d347gx6stktn</p><p><small>Oct <var data-var='date'> 4</var>, <var data-var='time'>08:01</var> PDT</small><br><strong>Investigating</strong> - We are currently investigating this issue.</p>tag:status.faros.ai,2005:Incident/186831592023-10-03T17:37:48-07:002023-10-03T17:42:39-07:00GraphQL API is returning "500 Internal Server Error"<p><small>Oct <var data-var='date'> 3</var>, <var data-var='time'>17:37</var> PDT</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Oct <var data-var='date'> 3</var>, <var data-var='time'>17:35</var> PDT</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Oct <var data-var='date'> 3</var>, <var data-var='time'>17:26</var> PDT</small><br><strong>Identified</strong> - The issue has been identified and a fix is being implemented.</p><p><small>Oct <var data-var='date'> 3</var>, <var data-var='time'>17:22</var> PDT</small><br><strong>Investigating</strong> - We are currently investigating this issue.</p>tag:status.faros.ai,2005:Incident/185697102023-09-22T18:00:22-07:002023-09-22T18:00:22-07:00Planned Maintenance<p><small>Sep <var data-var='date'>22</var>, <var data-var='time'>18:00</var> PDT</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Sep <var data-var='date'>22</var>, <var data-var='time'>17:00</var> PDT</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Sep <var data-var='date'>21</var>, <var data-var='time'>12:52</var> PDT</small><br><strong>Scheduled</strong> - We will be undergoing scheduled maintenance during this time.</p>tag:status.faros.ai,2005:Incident/185362822023-09-18T14:21:33-07:002023-09-18T14:21:33-07:00Degraded API performance<p><small>Sep <var data-var='date'>18</var>, <var data-var='time'>14:21</var> PDT</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Sep <var data-var='date'>18</var>, <var data-var='time'>13:19</var> PDT</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Sep <var data-var='date'>18</var>, <var data-var='time'>10:45</var> PDT</small><br><strong>Identified</strong> - The issue has been identified and a fix is being implemented.</p><p><small>Sep <var data-var='date'>18</var>, <var data-var='time'>08:56</var> PDT</small><br><strong>Investigating</strong> - We are currently investigating this issue.</p>tag:status.faros.ai,2005:Incident/182124542023-08-24T06:00:20-07:002023-08-24T06:00:20-07:00Planned Maintenance<p><small>Aug <var data-var='date'>24</var>, <var data-var='time'>06:00</var> PDT</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Aug <var data-var='date'>24</var>, <var data-var='time'>05:00</var> PDT</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Aug <var data-var='date'>21</var>, <var data-var='time'>10:23</var> PDT</small><br><strong>Scheduled</strong> - We will be undergoing scheduled maintenance during this time.</p>tag:status.faros.ai,2005:Incident/179153612023-07-20T13:17:19-07:002023-07-20T13:54:41-07:00Faros App offline<p><small>Jul <var data-var='date'>20</var>, <var data-var='time'>13:17</var> PDT</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Jul <var data-var='date'>20</var>, <var data-var='time'>12:52</var> PDT</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Jul <var data-var='date'>20</var>, <var data-var='time'>12:50</var> PDT</small><br><strong>Update</strong> - We are continuing to investigate this issue.</p><p><small>Jul <var data-var='date'>20</var>, <var data-var='time'>12:29</var> PDT</small><br><strong>Investigating</strong> - We are currently investigating this issue.</p>tag:status.faros.ai,2005:Incident/178900772023-07-18T07:30:00-07:002023-07-18T08:12:07-07:00Elevated Application and Dashboard latencies<p><small>Jul <var data-var='date'>18</var>, <var data-var='time'>07:30</var> PDT</small><br><strong>Resolved</strong> - Elevated Application and Dashboard latencies were elevated between 8:30pm 7/17 and 7:30am 7/18 PDT.</p>tag:status.faros.ai,2005:Incident/178263052023-07-13T18:00:54-07:002023-07-13T18:00:54-07:00Planned Maintenance<p><small>Jul <var data-var='date'>13</var>, <var data-var='time'>18:00</var> PDT</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Jul <var data-var='date'>13</var>, <var data-var='time'>17:00</var> PDT</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Jul <var data-var='date'>11</var>, <var data-var='time'>10:35</var> PDT</small><br><strong>Scheduled</strong> - We will be undergoing scheduled maintenance during this time.</p>tag:status.faros.ai,2005:Incident/176322112023-06-20T09:20:31-07:002023-06-20T09:20:31-07:00Slow load times<p><small>Jun <var data-var='date'>20</var>, <var data-var='time'>09:20</var> PDT</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Jun <var data-var='date'>20</var>, <var data-var='time'>09:13</var> PDT</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Jun <var data-var='date'>20</var>, <var data-var='time'>08:58</var> PDT</small><br><strong>Investigating</strong> - We are currently investigating this issue.</p>tag:status.faros.ai,2005:Incident/174468452023-06-01T14:37:06-07:002023-06-01T14:37:06-07:00API calls failing<p><small>Jun <var data-var='date'> 1</var>, <var data-var='time'>14:37</var> PDT</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Jun <var data-var='date'> 1</var>, <var data-var='time'>14:36</var> PDT</small><br><strong>Update</strong> - We have identified the affected component and have services have been returned to a functioning state.</p><p><small>Jun <var data-var='date'> 1</var>, <var data-var='time'>14:28</var> PDT</small><br><strong>Identified</strong> - The issue has been identified and a fix is being implemented.</p>tag:status.faros.ai,2005:Incident/173042122023-05-22T18:00:18-07:002023-05-22T18:00:18-07:00Planned Maintenance<p><small>May <var data-var='date'>22</var>, <var data-var='time'>18:00</var> PDT</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>May <var data-var='date'>22</var>, <var data-var='time'>17:00</var> PDT</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>May <var data-var='date'>19</var>, <var data-var='time'>09:46</var> PDT</small><br><strong>Scheduled</strong> - We will be undergoing scheduled maintenance during this time.</p>