[{"attributes":{"type":"Incident__c","url":"/services/data/v48.0/sobjects/Incident__c/a9C1Y0000004E65UAE"},"Id":"a9C1Y0000004E65UAE","Duration__c":184,"Log__c":"Okta is currently experiencing an issue with the service in US Cell 3. User Imports from Workday are resulting in users in No Password state. We are actively investigating and will update this message with more information as soon as we have it.\r\n\r\nResolved: Okta has resolved the issue where Workday imports for some Orgs changed users to a No Password state resulting in those users being unable to authenticate. Root Cause Analysis will be provided here within 2 business days.\r\n\r\nComplete Root Cause Analysis may be found [Here.](https://status.okta.com/#incident/a9C1Y0000004E65UAE)","Status__c":"Resolved","Start_Date__c":"2019-12-04","End_Date__c":"2020-12-04","Last_Updated__c":"2020-02-10T23:25:52.000+0000","LastModifiedDate":"2020-02-10T23:25:52.000+0000","Category__c":"Service Disruption","Size__c":"Large","Impacted_Cells__c":"okta.com:3","Impacted_Audience__c":"Admin;End user","Service_Feature__c":"Core Service","Start_Time__c":"2019-12-04T21:06:00.000+0000","CreatedDate":"2019-12-04T21:07:44.000+0000","CurrencyIsoCode":"USD"},{"attributes":{"type":"Incident__c","url":"/services/data/v48.0/sobjects/Incident__c/a9C1Y0000004E8uUAE"},"Id":"a9C1Y0000004E8uUAE","Duration__c":321,"Log__c":"Okta is currently experiencing an issue with the service in US Cell 1. AD Agent and IWA agent states may not be correctly reflected. We are actively investigating and will update this message with more information as soon as we have it.\r\n\r\nResolved: Okta has resolved an issue in US Cell 1 impacting AD/LDAP Agent state reporting. All on-line AD/LDAP Agents are now correctly reporting ACTIVE status. Additionally, some customers where all AD/LDAP Agents were marked NOT_CONNECTED should now see successful imports. Additional root cause information will be provided within 48 hours.\r\n\r\nRoot Cause Analysis: On January 22nd at approximately ‪11:39am PST‬ Okta’s monitoring systems detected internal server errors in US Cell 1. The errors resulted in some customers' AD and LDAP agents being reported as NOT_CONNECTED to Okta Admins, even though the agents were functioning normally. Additionally, customers where all AD Agents were marked offline and who had Desktop Single Sign-On (IWA) configured using automatic fail-over may have experienced IWA failures. Okta implemented mitigation steps and the errors subsided at approximately ‪12:06pm PST‬. Okta subsequently corrected agent status for all customers by approximately ‪5:00pm PST‬.\r\n\r\nUpon investigation, Okta determined that a mistake was made when updating references to the database system during the execution of a planned routine database maintenance, This misconfiguration caused a very small number of requests to return an HTTP 500 error code and also resulted in the outdated agent status information.\r\n\r\nOkta has updated run-books and is improving the automation of maintenance verification procedures to prevent future occurrences. Okta is also investigating detection of agent status check job failures to prevent incorrect status indication to administrators.","Status__c":"Resolved","Start_Date__c":"2020-01-22","End_Date__c":"2020-01-22","Last_Updated__c":"2020-01-27T20:18:33.000+0000","LastModifiedDate":"2020-01-27T20:18:33.000+0000","Category__c":"Service Degradation","Size__c":"Large","Impacted_Cells__c":"okta.com:1","Impacted_Audience__c":"End user","Service_Feature__c":"Core Service","Start_Time__c":"2020-01-22T23:33:00.000+0000","CreatedDate":"2020-01-22T23:35:32.000+0000","CurrencyIsoCode":"USD"},{"attributes":{"type":"Incident__c","url":"/services/data/v48.0/sobjects/Incident__c/a9C1Y0000004E7hUAE"},"Id":"a9C1Y0000004E7hUAE","Duration__c":59,"Log__c":"Okta had detected an issue causing increased error rates and response times for requests to OK4. The incident has been resolved. We implemented mitigating steps to return the service to normal as quickly as possible.\r\n\r\nResolved: An issue causing increased errors and response times for end-users and administrators in OK4 was resolved. During this event, End-Users and Admins may have encountered errors. Additional root cause information will provided within 48 hours.\r\n\r\nRoot Cause Analysis: Full RCA details for this event can be found [here](https://status.okta.com/#incident/a9C1Y0000004E7hUAE).","Status__c":"Resolved","Start_Date__c":"2020-01-05","End_Date__c":"2020-01-05","Last_Updated__c":"2020-01-08T14:21:11.000+0000","LastModifiedDate":"2020-01-08T14:21:11.000+0000","Category__c":"Service Disruption","Size__c":"Large","Impacted_Cells__c":"OK4","Service_Feature__c":"Core Service","Start_Time__c":"2020-01-05T10:16:00.000+0000","CreatedDate":"2020-01-05T10:30:02.000+0000","CurrencyIsoCode":"USD"},{"attributes":{"type":"Incident__c","url":"/services/data/v48.0/sobjects/Incident__c/a9C1Y0000004E73UAE"},"Id":"a9C1Y0000004E73UAE","Duration__c":107,"Log__c":"Okta is currently experiencing an issue with the service. We are actively investigating and will update this message with more information as soon as we have it.\r\n\r\nResolved: Okta has resolved the issue. All systems are operational issue has been mitigated. Okta will provide an RCA within 2 Business Days.\r\n\r\nRoot Cause Analysis: On December 16, 2019 at approximately 6:22 AM PST, Okta detected a degradation in application performance in US Cell 4. As a result, customers may have experienced:\r\n- Longer than normal response times\r\n- A small percentage (0.25%) experienced sporadic read-only errors\r\n- Admin users may have intermittently seen the read-only banner\r\n\r\nUpon investigation, Okta determined that the root cause was unusual database activity required to process a significant number of group rules and assignments. Although our throttling and rate limiting mechanisms engaged, a portion of requests saw additional latency due to the increased utilization of our database tier while the load was being redistributed.\r\nAt approximately 6:48 AM PST, Okta began to redistribute requests across the cell’s database cluster to reduce load and contention on the impacted database. By 8:09 AM PST, normal service performance was fully restored.\r\n\r\nOkta is implementing additional monitoring, optimization, and mitigation strategies to prevent similar issues in the future. Okta is also investigating asynchronous processing for group rules and performance improvements for group deletions to prevent system slowdown.","Status__c":"Resolved","Start_Date__c":"2019-12-16","End_Date__c":"2019-12-16","Last_Updated__c":"2019-12-19T02:29:21.000+0000","LastModifiedDate":"2019-12-19T02:29:21.000+0000","Category__c":"Service Degradation","Size__c":"Large","Incident_Title__c":"Performance Degradation on Cell OK4","Impacted_Cells__c":"okta.com:4","Impacted_Audience__c":"Admin;API Products;End user","Service_Feature__c":"Core Service","Start_Time__c":"2019-12-16T15:31:00.000+0000","CreatedDate":"2019-12-16T15:32:34.000+0000","CurrencyIsoCode":"USD"},{"attributes":{"type":"Incident__c","url":"/services/data/v48.0/sobjects/Incident__c/a9C1Y0000004E6jUAE"},"Id":"a9C1Y0000004E6jUAE","Duration__c":78,"Log__c":"Okta experienced an issue with Agentless Desktop SSO for a subset of ORGs. We are actively investigating and will update this message with more information as soon as we have it.\r\n\r\nResolved: Okta has resolved an issue where a subset of customers were unable to authenticate successfully via Agentless Desktop SSO in US Cell 1, US Cell 11, and EMEA Cell 1. Impacted customers may be required to clear their DNS cache to fully restore end user normalcy. Root Cause Analysis will be provided here within 2 business days.\r\n\r\nRoot Cause Analysis: On December 10th, 2019 at approximately 9:30 AM PST, Okta began investigating reports by customers of problems authenticating via Agent-less Desktop SSO in US Cell 1, US Cell 11, and EU Cell 1. \r\n\r\nOkta determined that during the implementation of new load-balancers on December 4th 2019 at 3:15pm PST, DNS entries for Kerberos endpoints were not updated correctly for the impacted cells. As a result, Agent-less Desktop SSO traffic was still directed to the previous load balancers which were no longer working as intended, resulting in requests that timed out and user browser sessions being redirected to Okta login screens. While affected users were able to login, they had to type in their credentials instead of being logged in automatically.\r\n\r\nAt approximately 10:48 AM, Okta updated the affected entries to reference the new load-balancers and service was restored. Some clients may have had to refresh DNS caches to restore service more quickly.\r\n\r\nOkta is working to implement additional monitoring and automation of changes to DNS records to prevent similar issues during future updates.","Status__c":"Resolved","Start_Date__c":"2019-12-10","End_Date__c":"2019-12-10","Last_Updated__c":"2019-12-16T08:19:31.000+0000","LastModifiedDate":"2019-12-16T08:19:31.000+0000","Category__c":"Service Degradation","Size__c":"Large","Impacted_Cells__c":"okta-emea.com:1;okta.com:1;okta.com:11","Impacted_Audience__c":"End user","Service_Feature__c":"Core Service","Start_Time__c":"2019-12-10T18:05:00.000+0000","CreatedDate":"2019-12-10T18:07:38.000+0000","CurrencyIsoCode":"USD"},{"attributes":{"type":"Incident__c","url":"/services/data/v48.0/sobjects/Incident__c/a9C1Y0000004E5WUAU"},"Id":"a9C1Y0000004E5WUAU","Duration__c":151,"Log__c":"Okta is currently experiencing an issue with job processing for a subset of customers in US Cell 7. We are actively investigating and will update this message with more information as soon as we have it.\r\n\r\nResolved: An issue impacting job processing for a subset of customers within US Cell 7 has been resolved. The remaining queued jobs are processing at full capacity and we will continue to monitor. Root Cause Analysis is underway and we will post full RCA details here within 48 hours.\r\n\r\nRoot Cause Analysis: On December 2nd at approximately 7:45AM (PST), Okta detected a slowdown with processing import jobs on US Cell 7 and subsequently determined that there was an unusual and significant increase in asynchronous job workload which resulted in a backup of job processing.\r\n\r\nOkta increased system resources to expedite the processing of pending jobs. This mitigated the issue and normal functionality was fully restored by ‪10:16 AM.\r\n\r\nOkta is working optimize the current implementation of group application assignments to batch and de-duplicate assignments. Okta is also adding monitoring and controls to detect and mitigate similar issues in the future","Status__c":"Resolved","Start_Date__c":"2019-12-02","End_Date__c":"2019-12-02","Last_Updated__c":"2019-12-08T03:03:48.000+0000","LastModifiedDate":"2019-12-08T03:03:48.000+0000","Category__c":"Service Degradation","Size__c":"Large","Impacted_Cells__c":"okta.com:7","Impacted_Audience__c":"Admin","Service_Feature__c":"Core Service","Start_Time__c":"2019-12-02T16:34:00.000+0000","CreatedDate":"2019-12-02T16:34:59.000+0000","CurrencyIsoCode":"USD"},{"attributes":{"type":"Incident__c","url":"/services/data/v48.0/sobjects/Incident__c/a9C1Y0000004E5RUAU"},"Id":"a9C1Y0000004E5RUAU","Duration__c":342,"Log__c":"Okta is currently experiencing an issue with Advanced Server Access. Administrators may have trouble accessing the dashboard, and users may experience errors with SSO or while attempting to authenticate to servers. We are actively investigating and will update this message with more information as soon as we have it.\r\n\r\nResolved: An issue impacting Okta Advanced Server Access has been resolved. During the event, Administrators may have experienced errors accessing the ASA dashboard. Additionally, users may have experienced authentication errors. Okta applied several changes to mitigate the issue and continues to monitor and investigate. Additional root cause information will be provided within 48 business hours.\r\n\r\nRoot Cause Analysis: On Saturday, November 23rd at approximately 1:05 PM PDT, Okta detected database resource contention within our Advanced Server Access production environment, resulting in timeout errors for users who were accessing the Advanced Server Access dashboard or were attempting to authenticate to servers.\r\n\r\nAfter investigating the issue, at approximately 3:30 PM PDT, Okta took actions to mitigate the resource contention issue by blocking customer API requests for adding groups to projects and synchronizing data to servers. Most Advanced Server Access functionality was restored through this action. Upon further investigation, Okta identified an additional inefficiency in our synchronous logic, and made a change to asynchronous background job processing.\r\n\r\nFollowing our investigation and mitigation actions, Okta re-enabled the API for synchronizing data to servers at approximately 6:47 PM PDT. Okta also re-enabled the API for adding groups to projects, fully restoring all services at this time.\r\n\r\nOkta's has implemented changes to mitigate the impact of similar issues in the future, and is continuing its full Root Cause Analysis of the incident,. We expect to be able to provide further details and future preventative actions by December 4, 2019. Additional RCA details will be posted here once this is investigation is complete.","Status__c":"Resolved","Start_Date__c":"2019-11-23","End_Date__c":"2019-11-23","Last_Updated__c":"2019-12-03T21:49:47.000+0000","LastModifiedDate":"2019-12-03T21:49:47.000+0000","Category__c":"Service Disruption","Size__c":"Large","Impacted_Cells__c":"EU1;OK1;OK2;OK3;OK4;OK5;OK6;Preview;okta-emea.com:1;okta.com:1;okta.com:2;okta.com:3;okta.com:4;okta.com:5;okta.com:6;okta.com:7;oktapreview.com:1;oktapreview.com:2;OK7;Preview2;okta.com:8;okta.com:9;okta.com:10;okta.com:11","Impacted_Audience__c":"Admin;End user","Service_Feature__c":"Core Service","Start_Time__c":"2019-11-23T23:06:00.000+0000","CreatedDate":"2019-11-23T23:07:45.000+0000","CurrencyIsoCode":"USD"},{"attributes":{"type":"Incident__c","url":"/services/data/v48.0/sobjects/Incident__c/a9C1Y0000004E4EUAU"},"Id":"a9C1Y0000004E4EUAU","Duration__c":684,"Log__c":"Okta customers in all cells that leverage Zoom’s integration for provisioning are experiencing errors while updating users in Zoom. We are actively working with Zoom to identify the root cause and identify a solution.\r\n\r\nResolved: An issue impacting user provisioning updates to the Zoom application within Okta has been resolved. During the event, user updates to the Zoom application may have encountered errors if the provisioned user already existed within Zoom. Okta worked with Zoom to identify the root cause of the issue and applied a hot-fix to all production cells to mitigate the provisioning errors while Zoom worked to deploy a permanent fix. Additional root cause information will be provided within 48 business hours.\r\n\r\nRoot Cause Analysis: On Nov 19th at approximately 8:53AM PDT, customers reported Zoom provisioning errors when updating user’s profiles.\r\n\r\nUpon investigation, Okta determined that the issue was caused by a change in the behavior of an existing Zoom Provisioning API which was introduced on November 17. This change was implemented by Zoom to enable support for changing user's email addresses through the Profile Update request. The change inadvertently caused profile updates to fail, whenever the email has not been changed.\r\n\r\nOn Nov 19th at approximately 8:17 PM PDT Okta released a work around to address the issue. This update was deployed to all production cells and service was fully restored.\r\n\r\nOkta is working closely with Zoom for a fix, as well as, understanding and mitigating the re-occurrence for this type of failure","Status__c":"Resolved","Start_Date__c":"2019-11-19","End_Date__c":"2019-11-19","Last_Updated__c":"2019-11-22T17:31:40.000+0000","LastModifiedDate":"2019-11-22T17:31:40.000+0000","Category__c":"Feature Disruption","Size__c":"Large","Impacted_Cells__c":"EU1;OK1;OK2;OK3;OK4;OK5;OK6;Preview;okta-emea.com:1;okta.com:1;okta.com:2;okta.com:3;okta.com:4;okta.com:5;okta.com:6;okta.com:7;oktapreview.com:1;oktapreview.com:2;OK7;Preview2;okta.com:8;okta.com:9;okta.com:10;okta.com:11","Impacted_Audience__c":"Admin;End user","Service_Feature__c":"Third Party","Start_Time__c":"2019-11-19T17:44:00.000+0000","CreatedDate":"2019-11-19T17:47:37.000+0000","CurrencyIsoCode":"USD"},{"attributes":{"type":"Incident__c","url":"/services/data/v48.0/sobjects/Incident__c/a9C1Y0000004E4JUAU"},"Id":"a9C1Y0000004E4JUAU","Duration__c":495,"Log__c":"Okta is investigating an issue with installations of Okta Mobile on Android. We will provide more information once available.\r\n\r\nResolved: Okta has addressed an issue on Okta Mobile for Android for the vast majority of Okta end users. When opening the Okta Mobile for Android application, end users may be logged out of their account and encounter the following message: “Detected possible malicious activity when establishing a secure connection to Okta”. Okta Mobile Android users on Android 6.0 or higher must update the Okta Mobile application to the latest version through the Google Play Store. Okta continues to investigate a solution for users running versions of Android below 6.0 as well as those relying on downloading the Okta Mobile APK from their Okta tenant. We apologize for the inconvenience. Additional root cause information will be provided within 48 business hours.\r\n\r\nRoot Cause Analysis: On Nov 19th at approximately 12:30 PM PDT, customers reported that when opening the Okta Mobile application for Android, end users were intermittently logged out of their account and encountered the following message: \r\n\r\n “Detected possible malicious activity when establishing a secure connection to Okta”.\r\n\r\nUpon investigation, Okta determined that the issue was caused by an update to the preferred authentication algorithm used in TLS by Okta Mobile’s error tracking provider. The new preferred authentication algorithm chosen by the provider became ECDSA, which was not allowed by the Okta Mobile Android application. Upon detecting a change in the encryption algorithm in use, Okta Mobile on Android is designed to present the aforementioned warning message.\r\n\r\nOn Nov 19th at approximately 8:45 PM PDT, Okta released Android Okta Mobile version 4.3.2, for Android versions 6.0 and above, and notified Org administrators through the Admin Console banner that users should update their Okta Mobile application to address the issue. Users running an older version of Android are encouraged to update their device to 6.0 or later and install the latest version of Okta Mobile.\r\n\r\nOkta is implementing additional monitoring and alerting to detect such issues earlier in the future. Additionally, the latest Android Okta Mobile version has expanded the set of authentication algorithms allowed and newer versions will include an improved design to prevent issues like this in the future.","Status__c":"Resolved","Start_Date__c":"2019-11-19","End_Date__c":"2019-11-19","Last_Updated__c":"2019-11-22T16:13:38.000+0000","LastModifiedDate":"2019-11-22T16:13:38.000+0000","Category__c":"Service Degradation","Size__c":"Large","Impacted_Cells__c":"EU1;OK1;OK2;OK3;OK4;OK5;OK6;Preview;okta-emea.com:1;okta.com:1;okta.com:2;okta.com:3;okta.com:4;okta.com:5;okta.com:6;okta.com:7;oktapreview.com:1;oktapreview.com:2;OK7;Preview2;okta.com:8;okta.com:9;okta.com:10;okta.com:11","Impacted_Audience__c":"End user","Service_Feature__c":"Core Service","Start_Time__c":"2019-11-19T20:49:00.000+0000","CreatedDate":"2019-11-19T20:54:32.000+0000","CurrencyIsoCode":"USD"},{"attributes":{"type":"Incident__c","url":"/services/data/v48.0/sobjects/Incident__c/a9C1Y0000004E2wUAE"},"Id":"a9C1Y0000004E2wUAE","Duration__c":125,"Log__c":"Okta is currently experiencing an issue with the service. We are actively investigating and will update this message with more information as soon as we have it. Office 365 policies are denying access to users.\r\n\r\nResolved: \r\n\r\nOkta has successfully deployed and verified a fix across all Cells. The identified issue impacted a subset of the users in all cells using legacy protocol sign-on with Office 365 or service accounts. Additional root cause information will provided within 48 hours.\r\n\r\nRoot Cause Analysis: \r\n\r\nOn Nov 6th at approximately 9:30 AM PDT, Okta received reports from certain customers across multiple cells that some users were not able to log into the Okta Service via Office 365 clients using legacy protocols such as the Outlook Desktop App, and some service accounts used for automation.\r\n\r\nOnly end users using Outlook clients with an Exchange ActiveSync profile and without an enrolled second factor that are members of tenants with a configured MFA Enrollment Policy were impacted.\r\n\r\nThe pursuant investigation determined that a bug fix, applied in the 2019.10.2 weekly update, to address a gap in multi-factor enrollment during authentication, was the root cause. The fix was incorrectly scoped to include legacy Office 365 protocols.\r\n\r\nAt approximately 10:44 AM PDT, Okta reverted the change in US Cell 4 and confirmed that issue was resolved. The change was then subsequently reverted in all production. At approximately 11:35 AM, service was fully restored for all customers in all cells.\r\n\r\nAdditional testing and alerting to prevent and detect such issues will be implemented to prevent future occurrences.","Status__c":"Resolved","Start_Date__c":"2019-11-06","End_Date__c":"2019-11-06","Last_Updated__c":"2019-11-10T17:25:07.000+0000","LastModifiedDate":"2019-11-10T17:25:07.000+0000","Category__c":"Service Disruption","Size__c":"Large","Incident_Title__c":"Office 365 policies are deny access to users.","Impacted_Cells__c":"EU1;OK1;OK2;OK3;OK4;OK5;OK6;Preview;okta-emea.com:1;okta.com:1;okta.com:2;okta.com:3;okta.com:4;okta.com:5;okta.com:6;okta.com:7;oktapreview.com:1;oktapreview.com:2;OK7;Preview2;okta.com:8;okta.com:9;okta.com:10;okta.com:11","Impacted_Audience__c":"End user","Service_Feature__c":"Core Service","Start_Time__c":"2019-11-06T17:32:00.000+0000","CreatedDate":"2019-11-06T17:35:16.000+0000","CurrencyIsoCode":"USD"},{"attributes":{"type":"Incident__c","url":"/services/data/v48.0/sobjects/Incident__c/a9C1Y0000004DyLUAU"},"Id":"a9C1Y0000004DyLUAU","Duration__c":1620,"Log__c":"We are aware of slow dashboard load times on Google Chrome version 77. We are investigating this issue and will update this message with more information as soon as we have it.\r\n\r\n==========================\r\n\r\nResolved: The issue with Google Chrome version 77 (on both desktop and Android) which was due to a change introduced in Google Chrome version 77 is now resolved on all Cells. Additional root cause information will be provided within 48 business hours.\r\n\r\n==========================\r\n\r\nRoot Cause Analysis: On Sept 30th, at approximately 3:39PM PDT, customers reported experiencing longer than expected load times for application logos in the Okta end-user dashboard and catalog.\r\n\r\nUpon investigation, Okta determined users in all cells experienced the longer load times after updating their Chrome browser to version 77.0.3865.75 (Official Build) (64-bit). Users of browsers other than Chrome, and users of Chrome earlier than version 77, were not impacted. Okta discovered that Chrome version 77 caused a performance regression with some HTML Canvas operations when the hardware acceleration setting was available and enabled. This behavior is also discussed in the Chromium bug forum as Issue 1001845.\r\n\r\nAt approximately 8:40PM PDT on Sept 30th, Okta started the phased deployment of a workaround, per-cell basis, which was completed on all cells by 10/1/2019 at 6:40 PM PDT.\r\n\r\nDuring the incident, users were advised to workaround the issue by disabling Hardware Acceleration. This workaround is no longer necessary and Hardware Acceleration in Chrome can be re-enabled.","Status__c":"Resolved","Start_Date__c":"2019-09-30","End_Date__c":"2019-10-01","Last_Updated__c":"2019-10-04T02:09:43.000+0000","LastModifiedDate":"2019-10-04T02:09:43.000+0000","Category__c":"Service Degradation","Size__c":"Large","Incident_Title__c":"Page Load Degradation on Google Chrome version 77","Impacted_Cells__c":"okta-emea.com:1;okta.com:1;okta.com:2;okta.com:3;okta.com:4;okta.com:5;okta.com:6;okta.com:7;oktapreview.com:1;oktapreview.com:2","Impacted_Audience__c":"End user","Service_Feature__c":"Third Party","Start_Time__c":"2019-09-30T22:08:00.000+0000","CreatedDate":"2019-09-30T22:50:15.000+0000","CurrencyIsoCode":"USD"},{"attributes":{"type":"Incident__c","url":"/services/data/v48.0/sobjects/Incident__c/a9C1Y000000fxj2UAA"},"Id":"a9C1Y000000fxj2UAA","Duration__c":103,"Log__c":"Okta is currently experiencing an issue with new Developer tenant activation e-mails. In some use-cases, the activation link is present, but not visible within the activation email. We are actively investigating and will update this message with more information as soon as we have it.\r\n\r\nRESOLVED: Okta has successfully deployed and verified the fix for the issue with new Developer tenant activation e-mail in US Cell 7. Additional root cause information will provided within 48 hours.\r\n\r\nROOT CAUSE: On September 5th, at approximately 12:27 PM PDT Okta detected an issue in the Developer tenant's activation email which impacted developer org creation in US Preview 1 and US Cell 7.\r\n\r\nUpon investigation, Okta determined that a software change to the Developer Tenant Email Template caused the generated emails to have noticeable styling issues and, in some clients, this caused one-time temporary passwords to become unreadable.\r\n\r\nAt approximately 1:49 PM PDT Okta verified a fix and began deploying the change as a patch to the impacted cells. At approximately 2:13 PM PDT the fix was fully deployed and generated emails were sent with visible temporary passwords.\r\n\r\nAdditional testing and monitoring to detect and mitigate this scenario will be rolled out for affected environments to prevent future occurrences.","Status__c":"Resolved","Start_Date__c":"2019-09-05","End_Date__c":"2019-09-05","Last_Updated__c":"2019-09-10T06:20:27.000+0000","LastModifiedDate":"2019-09-10T06:20:27.000+0000","Category__c":"Service Degradation","Size__c":"Large","Impacted_Cells__c":"okta.com:7","Impacted_Audience__c":"Admin","Service_Feature__c":"Core Service","Start_Time__c":"2019-09-05T19:29:00.000+0000","CreatedDate":"2019-09-05T19:31:34.000+0000","CurrencyIsoCode":"USD"},{"attributes":{"type":"Incident__c","url":"/services/data/v48.0/sobjects/Incident__c/a9C1Y000000fxh6UAA"},"Id":"a9C1Y000000fxh6UAA","Duration__c":100,"Log__c":"An issue affecting GSuite's OAuth endpoints is preventing customers who use GSuite for authentication from accessing ASA's web interface or enrolled servers.\r\n\r\nGoogle has opened an incident to track this:\r\nhttps://status.cloud.google.com/incident/developers-console/19008\r\n\r\n============================\r\n\r\nResolved: On Aug 19th at approximately 12:30PM PDT, Okta noted an issue affecting GSuite's OAuth 2.0 endpoints preventing customers who use GSuite for authentication from accessing ASA's web interface or enrolled servers, Google imports, and SSO login to GSuite apps.\r\n\r\nGoogle is reporting the issue is resolved for OAuth 2.0. Okta will continue to track the status of this incident and explore possible workarounds.\r\n\r\n============================\r\n\r\nRoot Cause Analysis: On August 19th, at approximately 10:50 AM PDT Okta detected an increased number of failed Google OAuth 2.0 authentication requests and import failures. In addition to that, customers using GSuite as their main IdP could not authenticate via Advanced Server Access (ASA) to access their infrastructure.\r\n\r\nUpon investigation, Okta determined that Google had experienced a [global outage](https://status.cloud.google.com/incident/developers-console/19008) of their OAuth 2.0 endpoints. This prevented external services (ASA, Okta) from authenticating user sessions against Google as an IdP, and no workarounds were provided.\r\n\r\nAt approximately 12:30 PM PDT the issue with Google OAuth 2.0 endpoints was fully resolved and service was restored.","Status__c":"Resolved","Start_Date__c":"2019-08-19","End_Date__c":"2019-08-19","Last_Updated__c":"2019-08-22T05:26:49.000+0000","LastModifiedDate":"2019-08-22T05:26:49.000+0000","Category__c":"Service Disruption","Size__c":"Large","Impacted_Cells__c":"EU1;OK1;OK2;OK3;OK4;OK5;OK6;Preview;OK7;Preview2","Impacted_Audience__c":"Admin;End user","Service_Feature__c":"Third Party","Start_Time__c":"2019-08-19T19:23:57.000+0000","CreatedDate":"2019-08-19T19:23:57.000+0000","CurrencyIsoCode":"USD"},{"attributes":{"type":"Incident__c","url":"/services/data/v48.0/sobjects/Incident__c/a9C1Y000000fxgDUAQ"},"Id":"a9C1Y000000fxgDUAQ","Duration__c":41,"Log__c":"Okta is currently experiencing an issue with the service. We are actively investigating and will update this message with more information as soon as we have it.\r\n\r\nRESOLVED: An issue impacting syslog accessibility in all Okta Cells has been identified and fixed. Service has been restored to normal.\r\n\r\nROOT CAUSE ANALYSIS: On On July 11th, at approximately 10:47 AM PDT Okta detected System Logs access failures across all cells.\r\n\r\nUpon investigation, Okta determined that the issue was caused by an expired Certificate Revocation List (CRL), which had been updated in the certificate keystore repository, but had not been synced across System Logs storage infrastructure. This resulted in certificate validation failures when users attempted to access System Logs.\r\n\r\nAt 11:06 AM PDT, Okta updated the CRL across all cells. Errors started clearing as the change was applied and caches were progressively refreshed. At approximately 11:28 AM PDT, all errors had been cleared.\r\n\r\nOkta is implementing additional monitoring and alerting to further enhance detection of similar CRL errors in the future. We are also implementing additional process changes and tooling to prevent similar CRL-related certificate validation errors in the future.","Status__c":"Resolved","Start_Date__c":"2019-07-11","End_Date__c":"2019-07-11","Last_Updated__c":"2019-07-17T06:55:21.000+0000","LastModifiedDate":"2019-07-17T06:55:21.000+0000","Category__c":"Service Disruption","Size__c":"Medium","Impacted_Cells__c":"EU1;OK1;OK2;OK3;OK4;OK5;OK6;Preview;okta-emea.com:1;okta.com:1;okta.com:2;okta.com:3;okta.com:4;okta.com:5;okta.com:6;okta.com:7;oktapreview.com:1;oktapreview.com:2;OK7;Preview2","Impacted_Audience__c":"Admin","Service_Feature__c":"Core Service","Start_Time__c":"2019-07-11T18:17:00.000+0000","CreatedDate":"2019-07-11T18:18:41.000+0000","CurrencyIsoCode":"USD"},{"attributes":{"type":"Incident__c","url":"/services/data/v48.0/sobjects/Incident__c/a9C1Y000000fxWSUAY"},"Id":"a9C1Y000000fxWSUAY","Duration__c":306,"Log__c":"Okta is currently experiencing an issue with Cloud Desktop Single Sign-On when using Google Chrome, where attempting to sign-in results in an HTTP 400 response. <br><br>We have validated a fix and are in the process of deploying a hot fix to all cells to address this issue. <br><br>Cloud Desktop Single Sign-On is still available on Apple Safari and Internet Explorer.<br><br>Issue has been resolved in US Cell 4, US Cell 5, EMEA Cell 1, and US Cell 7.\r\n<br><br>\r\nResolved:An issue impacting Cloud Desktop Single Sign-On for users of Google Chrome has been resolved. During the event, End-Users may have experienced HTTP 400 responses, when attempting to login using Google Chrome. Additional root cause information will be provided within 48 business hours.\r\n<br><br>\r\nRoot Cause: On May 31st, at approximately 12:24 PM PDT Okta received reports from some customers receiving a '400 Bad Request' response while attempting to login with Agentless Desktop SSO using Chrome browsers.\r\n<br><br>\r\nUpon investigation, Okta determined that the issue impacted customers with either incorrect Chrome settings or network configuration (see https://help.okta.com/en/prod/Content/Topics/Directory/Configuring_Agentless_SSO.htm). Okta also determined that the issue had been exposed by the release of a software performance optimization which did not account for this possible browser mis-configuration; prior to the optimization, this mis-configuration would have resulted in users being redirected to the default login page.\r\n<br><br>\r\nGiven the impact and poor user experience, starting on May 31st at 2:30 PM PDT, Okta deployed a change that reverted behavior to the pre-optimization one. The deployment completed to all cells at approximately 5:30PM PDT.\r\n<br><br>\r\nOkta is implementing additional testing and monitoring to detect this condition in the future.","Status__c":"Resolved","Start_Date__c":"2019-05-31","End_Date__c":"2019-05-31","Last_Updated__c":"2019-06-05T06:31:07.000+0000","LastModifiedDate":"2019-06-05T06:31:07.000+0000","Category__c":"Service Disruption","Size__c":"Large","Impacted_Cells__c":"okta-emea.com:1;okta.com:1;okta.com:2;okta.com:3;okta.com:4;okta.com:5;okta.com:6;okta.com:7;oktapreview.com:1;oktapreview.com:2","Impacted_Audience__c":"End user","Service_Feature__c":"Core Service","Start_Time__c":"2019-05-31T19:52:42.000+0000","CreatedDate":"2019-05-31T19:52:42.000+0000","CurrencyIsoCode":"USD"},{"attributes":{"type":"Incident__c","url":"/services/data/v48.0/sobjects/Incident__c/a9C1Y000000fxVtUAI"},"Id":"a9C1Y000000fxVtUAI","Duration__c":173,"Log__c":"Okta is currently experiencing an issue in US Preview Cell 1. Users attempting to authenticate under certain CORS use-cases are experiencing HTTP 400 errors\r\n<BR><BR>\r\nRESOLVED: Okta has identified and resolved an issue resulting in HTTP 400 errors when users were attempting to authenticate under certain CORS use-cases with tenants in US Preview Cell 1. Additional root cause information will provided within 48 hours.\r\n<BR><BR>\r\nROOT CAUSE: On May 15th at approximately 1:42PM PDT, customers in US Preview Cell 2 experienced elevated errors when making HTTP OPTIONS-based requests, predominately against the \"/oauth2/v1/userinfo\" and \"/api/v1/sessions/*****\" endpoints. At 3:09PM PDT, Okta began to detect similar errors in US Preview Cell 1.\r\n<BR><BR>\r\nThe elevated error rate was determined to be due to a bug within a new framework used to determine request timeout configurations, and specifically impacted the HTTP OPTIONS method. The new timeout framework was disabled to mitigate the issue, and the service was fully restored at approximately 4:35PM PDT for US Preview Cell 1, and at 4:43PM PDT for US Preview Cell 2.\r\n<BR><BR>\r\nFollowing this incident, this bug in the new timeout framework has been addressed. Okta will be adding additional test procedures to detect this issue prior to re-enabling the framework.","Status__c":"Resolved","Start_Date__c":"2019-05-15","End_Date__c":"2019-05-15","Last_Updated__c":"2019-05-24T13:59:22.000+0000","LastModifiedDate":"2019-05-24T13:59:22.000+0000","Category__c":"Service Disruption","Size__c":"Large","Impacted_Cells__c":"oktapreview.com:1;oktapreview.com:2","Impacted_Audience__c":"API Products;End user","Service_Feature__c":"Core Service","Start_Time__c":"2019-05-15T23:31:01.000+0000","CreatedDate":"2019-05-15T23:31:01.000+0000","CurrencyIsoCode":"USD"},{"attributes":{"type":"Incident__c","url":"/services/data/v48.0/sobjects/Incident__c/a9C1Y000000fxVFUAY"},"Id":"a9C1Y000000fxVFUAY","Duration__c":22,"Log__c":"Okta is currently experiencing an issue on OK5. We are actively investigating and will update this message with more information as soon as we have it. Please visit <a href=\"https://status.okta.com\">https://status.okta.com</a> to view the current status.\r\n<br><br> \r\nResolved: On April 29th at 11:26AM PDT, Okta has resolved the issue on OK5 and is monitoring. Service has been restored to normal.\r\n\r\nRoot Cause: On Apr 29th at approximately 11:04AM PDT, Okta detected slightly elevated error rates in US Cell 5, pointing to intermittent unavailability of the Okta Administration dashboard. Investigation showed this to be an issue with the fail-over a database read replica after a hardware failure. Out of an abundance of caution, the application was briefly placed in [Read Only mode](https://support.okta.com/help/s/article/Read-only-Mode) at approximately 11:24AM PDT while the issue was addressed. The service was returned to active mode when the corrective procedure was completed at 11:26am PDT.\r\n<br><br> \r\nOkta is actively working with the infrastructure service provider to understand and mitigate the risk of this hardware failure. In addition, we are tuning the fail-over configuration of the read replica database.","Status__c":"Resolved","Start_Date__c":"2019-04-29","End_Date__c":"2019-04-29","Last_Updated__c":"2019-05-03T16:09:44.000+0000","LastModifiedDate":"2019-05-03T16:09:44.000+0000","Category__c":"Service Disruption","Size__c":"Medium","Impacted_Cells__c":"okta.com:5","Impacted_Audience__c":"Admin","Service_Feature__c":"Core Service","Start_Time__c":"2019-04-29T18:26:00.000+0000","CreatedDate":"2019-04-29T18:30:29.000+0000","CurrencyIsoCode":"USD"},{"attributes":{"type":"Incident__c","url":"/services/data/v48.0/sobjects/Incident__c/a9C1Y000000GpKxUAK"},"Id":"a9C1Y000000GpKxUAK","Duration__c":60,"Log__c":"Okta is currently experiencing an issue with Office 365 provisioning. We are actively investigating and will update this message with more information as soon as we have it.\r\n<br><br>\r\nResolved: On Feb 7th at approximately 4:30PM PDT, Okta began receiving reports from customers in US Cell 4 who were experiencing errors while attempting to provision users to Office 365 or while modifying Office 365 configuration settings. User authentication was not impacted at any point.\r\n<br><br>\r\nThe root cause was traced to an inability in some cases to read domain federation settings from Microsoft Office 365. This bug caused a failure to successfully provision users and save Office 365 app settings. Okta also determined that this issue was present in all cells, but only affected a subset of customers.\r\n<br><br>\r\nOnce the root cause was identified, Okta deployed a fix starting at 4:49pm PDT on Feb 7. at approximately 5:30pm PDT, the fix was successfully rolled out to all production cells, and functionality was fully restored.\r\n<br><br>\r\nAdditional monitoring to detect and mitigate this scenario has been rolled out for all environments. Testing will be augmented to prevent future occurrences.","Status__c":"Resolved","Start_Date__c":"2019-02-07","End_Date__c":"2019-02-07","Last_Updated__c":"2019-02-19T10:26:43.000+0000","LastModifiedDate":"2019-02-19T10:26:43.000+0000","Category__c":"Service Degradation","Size__c":"Large","Impacted_Cells__c":"EU1;OK1;OK2;OK3;OK4;OK5;OK6;Preview;okta-emea.com:1;okta.com:1;okta.com:2;okta.com:3;okta.com:4;okta.com:5;okta.com:6;okta.com:7;oktapreview.com:1;oktapreview.com:2;OK7;Preview2","Impacted_Audience__c":"Admin","Service_Feature__c":"Core Service","Start_Time__c":"2019-02-08T00:33:00.000+0000","CreatedDate":"2019-02-08T00:36:33.000+0000","CurrencyIsoCode":"USD"},{"attributes":{"type":"Incident__c","url":"/services/data/v48.0/sobjects/Incident__c/a9C1Y000000GpKsUAK"},"Id":"a9C1Y000000GpKsUAK","Duration__c":212,"Log__c":"Okta is currently experiencing an issue where errors have been reported with O365 provisioning.\r\n\r\nThe incident is now resolved - Resolved: The issue impacting office 365 users experiencing provisioning errors has been fixed in EU1, OP1, OK1, OK2, OK3, and OK5. Service has been restored to normal.\r\n\r\nResolved: On Feb 5th at approximately 4:10pm PDT, Okta began receiving sporadic reports from customers in US Cell 1 and EMEA Cell 1 who were experiencing errors while attempting to provision users to Office 365 or modifying Office365 configuration settings. User authentication was not impacted at any point.\r\nThe root cause was traced to a serialization bug that affected a portion of Office365 application tenants. In particular, those where federation was auto configured using the Okta Admin UI prior to 2016, or manually configured by running the Okta provided Azure AD powershell cmdlets.\r\n\r\nOnce the root cause was identified, Okta deployed a fix starting at 5:17pm PDT on Feb 5. At 7:42pm PDT, the fix was completely rolled out to all production cells, and functionality was fully restored.\r\n\r\nTesting and monitoring are being augmented to prevent future occurrences.","Status__c":"Resolved","Start_Date__c":"2019-02-05","End_Date__c":"2019-02-05","Last_Updated__c":"2019-02-08T10:39:35.000+0000","LastModifiedDate":"2019-02-08T10:39:35.000+0000","Category__c":"Service Degradation","Size__c":"Large","Impacted_Cells__c":"EU1;okta.com:1;okta.com:2;okta.com:3;okta.com:5;oktapreview.com:1","Service_Feature__c":"Core Service","Start_Time__c":"2019-02-06T00:36:00.000+0000","CreatedDate":"2019-02-06T00:39:40.000+0000","CurrencyIsoCode":"USD"},{"attributes":{"type":"Incident__c","url":"/services/data/v48.0/sobjects/Incident__c/a9C1Y000000GpJuUAK"},"Id":"a9C1Y000000GpJuUAK","Duration__c":977,"Log__c":"Okta is currently experiencing an issue with CORS authentication in Okta Preview Cells; customers using OIDC applications or API AM might be impacted. Okta is actively working on resolving the issue.\r\n<br><br> \r\nRoot Cause Analysis: On December 12th, an issue was introduced in Okta Preview cells which impacted the use of the Auth SDKs. Specifically, customers using version 2.0 or higher of Okta’s JavaScript SDK AuthJS (@okta/okta-auth-js) to leverage OAuth 2.0 may have experienced authentication failures starting at approximately 2:21pm PST. Additionally, customers using Okta’s Angular, React and Vue integration libraries and OAuth 2.0 in Preview cells were also impacted by this issue. The degree of impact varied depending on each customers’ implementation and client-side caching. \r\n<br><br> \r\nThe root cause was traced to an unexpected side effect from a fix applied to our CORS/TrustedOrigins framework. The issue was fully resolved on Dec 13th by 6:26am PST in US Preview Cell 1 and by 6:38am PST in EU Preview Cell 2. To prevent a similar re-occurrence, the test automation framework gap has been updated and existing test suites are being enhanced to identify such errors in the future..","Status__c":"Resolved","Start_Date__c":"2018-12-13","End_Date__c":"2018-12-13","Last_Updated__c":"2018-12-17T07:39:46.000+0000","LastModifiedDate":"2018-12-17T07:39:46.000+0000","Category__c":"Service Disruption","Size__c":"Large","Impacted_Cells__c":"oktapreview.com:1;oktapreview.com:2","Impacted_Audience__c":"API Products;End user","Service_Feature__c":"Core Service","Start_Time__c":"2018-12-13T12:57:00.000+0000","CreatedDate":"2018-12-13T13:04:33.000+0000","CurrencyIsoCode":"USD"},{"attributes":{"type":"Incident__c","url":"/services/data/v48.0/sobjects/Incident__c/a9C0Z0000008PW7UAM"},"Id":"a9C0Z0000008PW7UAM","Duration__c":216,"Log__c":"We observed some delays or failures in the delivery of outbound SMS to the AT&T network in the US. SMS sent to other networks was unaffected. Most end users were able to succeed with a retry. We actively worked with our providers to resolve the issue and will provide an update when we have further information. Okta recommended affected AT&T mobile customers use an alternate second factor to work around the issue.\r\n\r\n\r\nResolved: Okta received reports of delays and failures in the delivery of SMS for multi-factor authentication. Upon further investigation, it was determined that the issue was caused by an on-going AT&T cellular network incident. Okta worked with its service providers to raise the issue and worked with customers to workaround the the issue until service was fully restored by AT&T. Users who were not on the AT&T cellular network or that leveraged other authentication factors were not affected throughout the incident.","Status__c":"Resolved","Start_Date__c":"2018-11-11","Last_Updated__c":"2018-11-14T19:15:28.000+0000","LastModifiedDate":"2018-11-14T19:15:28.000+0000","Category__c":"Service Degradation","Size__c":"Large","Impacted_Cells__c":"okta-emea.com:1;okta.com:1;okta.com:2;okta.com:3;okta.com:4;okta.com:5;okta.com:6;okta.com:7;oktapreview.com:1;oktapreview.com:2","Impacted_Audience__c":"End user","Service_Feature__c":"Third Party","Start_Time__c":"2018-11-11T23:36:00.000+0000","CreatedDate":"2018-11-11T23:41:45.000+0000","CurrencyIsoCode":"USD"},{"attributes":{"type":"Incident__c","url":"/services/data/v48.0/sobjects/Incident__c/a9C0Z0000008PViUAM"},"Id":"a9C0Z0000008PViUAM","Duration__c":1384,"Log__c":"On Oct 23rd at approximately 9:15pm PDT, Okta began receiving reports from customers who were experiencing errors while attempting to provision users to Office 365 services hosted by Microsoft. User authentication was not affected.\r\n\r\n==========\r\n\r\nResolved: On Oct 24th at 9:49am PDT, Okta Engineering identified an incompatibility with the latest release of Office 365 in a subset of Office 365 tenants. The incompatibility interrupted completion of the user provisioning process. A fix was identified and rolled out to production cells, and user provisioning functionality was fully restored at approximately 8:19pm PDT.\r\n<br><br> \r\nOkta is evaluating this incompatibility condition to ensure such occurrences are monitored and detected in the future.\r\n<br><br> \r\nEvent Root Cause Analysis details can be found [here](https://support.okta.com/help/Documentation/Knowledge_Article/Root-Cause-Analysis-10232018-Office-365-Provisioning-Failuers).","Status__c":"Resolved","Start_Date__c":"2018-10-24","End_Date__c":"2018-10-24","Last_Updated__c":"2018-10-29T08:50:00.000+0000","LastModifiedDate":"2018-10-29T08:50:00.000+0000","Category__c":"Service Degradation","Size__c":"Large","Impacted_Cells__c":"okta.com:1;okta.com:2;okta.com:3;okta.com:4;okta.com:6;oktapreview.com:1","Impacted_Audience__c":"Admin","Service_Feature__c":"Third Party","Start_Time__c":"2018-10-24T17:16:00.000+0000","CreatedDate":"2018-10-24T17:18:44.000+0000","CurrencyIsoCode":"USD"},{"attributes":{"type":"Incident__c","url":"/services/data/v48.0/sobjects/Incident__c/a9C0Z0000008PVOUA2"},"Id":"a9C0Z0000008PVOUA2","Duration__c":12,"Log__c":"Okta is currently experiencing an issue with the service. We are actively investigating and will update this message with more information as soon as we have it.\r\n\r\n\r\nResolved: On October 15th, beginning at 3:58 PM PDT, Okta experienced a service disruption in US Cell 6. The service disruption occurred during two distinct windows. Between 3:58 PM PDT and 4:05 PM PDT, approximately 15% of incoming end user requests to US Cell 6 experienced latency or errors. Between 4:28 PM PDT and 4:33 PM PDT, approximately 9% of end-user requests to US Cell 6 experienced latency or errors. The service disruption was resolved and the service was returned to normal at 4:33pm PDT.\r\n\r\nThe root cause of the service disruptions was traced to a significant and abnormal increase in requests that activated Okta's rate limiting framework but eventually resulted in higher than normal CPU utilization within our infrastructure. Starting at 3:57, Okta's monitoring services detected the increase in abnormal traffic, and automated service protections were employed to mitigate the impact, however, additional actions were needed to reroute and isolate the offending traffic.\r\n\r\nFollowing the service disruption, Okta reviewed the incident response and has identified and plans to implement additional automated and manual incident response process improvements and infrastructure changes to further minimize impact or prevent similar incidents in the future.","Status__c":"Resolved","Start_Date__c":"2018-10-15","End_Date__c":"2018-10-15","Last_Updated__c":"2018-10-18T04:46:11.000+0000","LastModifiedDate":"2018-10-18T04:46:11.000+0000","Category__c":"Service Disruption","Size__c":"Small","Impacted_Cells__c":"okta.com:6","Impacted_Audience__c":"End user","Service_Feature__c":"Core Service","Start_Time__c":"2018-10-15T23:09:00.000+0000","CreatedDate":"2018-10-15T23:09:28.000+0000","CurrencyIsoCode":"USD"},{"attributes":{"type":"Incident__c","url":"/services/data/v48.0/sobjects/Incident__c/a9C0Z0000008PUVUA2"},"Id":"a9C0Z0000008PUVUA2","Duration__c":135,"Log__c":"Resolved: On September 17th, at 4:40pm PDT, an issue began surfacing which impacted end users and admins attempting to authenticate to Microsoft Office 365 in US Cell 1.\r\n\r\nThe root cause was a manual error which occurred during normal preparation steps for the deployment of the weekly release. Specifically, the preparation step which applies new app version information into a pre-production environment for final release validation resulted in a new app version definition of Office 365 to be applied to US Cell 1. Because the Office 365 app version definition was then ahead of the US Cell 1 production environment, some authentication requests received HTTP 500 error responses. At the highest level of impact, 42% of browser initiated, and 7% of thick client (non-browser) initiated, authentication requests to O365 in US Cell 1 received a HTTP 500 error response. There was no impact to those with active sessions.\r\nOnce identified, a strategy to safely roll-back the updated app version definition was confirmed and initiated. By 6:14pm PDT, the issue was resolved for a subset of customers and by 6:55pm PDT the issue was fully resolved.\r\n\r\nTo prevent a future reoccurrence, actions are being taken in multiple phases. Immediately, a procedural change was implemented to accelerate internal escalation and recovery procedures for similar situations. Within the next few weeks, improvements will be made to the management of alerts enabling faster error identification, and to automation tooling to automatically prevent the application of incompatible app version definitions.","Status__c":"Resolved","Start_Date__c":"2018-09-17","Last_Updated__c":"2018-09-20T03:44:40.000+0000","LastModifiedDate":"2018-09-20T03:44:40.000+0000","Category__c":"Service Degradation","Size__c":"Large","Impacted_Cells__c":"okta.com:1","Impacted_Audience__c":"End user","Service_Feature__c":"Core Service","Start_Time__c":"2018-09-18T00:57:00.000+0000","CreatedDate":"2018-09-18T00:58:30.000+0000","CurrencyIsoCode":"USD"},{"attributes":{"type":"Incident__c","url":"/services/data/v48.0/sobjects/Incident__c/a9C0Z0000008PTwUAM"},"Id":"a9C0Z0000008PTwUAM","Duration__c":371,"Log__c":"Resolved: Okta experienced an issue when users attempted to navigated to their *.oktapreview.com tenant within Google Chrome or Firefox. When populating credentials, users may have experienced the following warning: \"Deceptive site ahead\". Okta worked with Google to implement mitigating steps to resolve the issue.\r\n\r\n_____\r\n\r\n\r\nRoot Cause Analysis: On September 5th, 2018 at approximately 1:12pm PDT, an issue impacting Preview cells (Preview Cell 1, Preview Cell 2). Customer's end users and administrators using Chrome or Firefox browsers with Safe Browsing enabled may have been prompted with an anti-phishing warning message while attempting to login into oktapreview subdomains. Similar login requests via other browsers, and all requests to other end-user and API endpoints were not impacted.\r\n\r\nThe root cause was traced to a false-positive determination by the [Google Safe Browsing](https://safebrowsing.google.com/) unsafe site detection service integrated into Chrome and Firefox, which reported any *.oktapreview.com/login was a likely phishing page.\r\n\r\nThe issue was fully resolved by 7:23pm PDT following collaboration with Google’s anti-phishing team. To prevent a future recurrence, actions are underway to engage with top browser and associated service providers to partner on safeguards such as whitelists and early warning notifications.","Status__c":"Resolved","Start_Date__c":"2018-09-04","End_Date__c":"2018-09-04","Last_Updated__c":"2018-09-12T23:03:00.000+0000","LastModifiedDate":"2018-09-12T23:03:00.000+0000","Category__c":"Service Disruption","Size__c":"Large","Impacted_Cells__c":"oktapreview.com:1;oktapreview.com:2","Impacted_Audience__c":"Admin;End user","Service_Feature__c":"Third Party","Start_Time__c":"2018-09-04T21:49:00.000+0000","CreatedDate":"2018-09-04T21:57:16.000+0000","CurrencyIsoCode":"USD"},{"attributes":{"type":"Incident__c","url":"/services/data/v48.0/sobjects/Incident__c/a9C0Z0000008PTrUAM"},"Id":"a9C0Z0000008PTrUAM","Duration__c":4271,"Log__c":"Resolved: An issue impacting authentication and system responsiveness for a subset of end-users and administrators accessing Microsoft Office 365 has subsided. Further investigation of the issue found the errors were correlated with a service degradation reported by Microsoft (Incident MO147606). Further information is available from Microsoft via https://status.office365.com.\r\n\r\n-------------------------------\r\n\r\nRoot Cause Analysis: On Sept 4th, 2018, at approximately 6:00am PDT, Okta began receiving reports that end-users were experiencing delays and errors while attempting to authenticate to Office 365 and related services hosted by Microsoft. Upon further investigation the service disruption began at approximately 2:29am PDT and appeared to be localized to customers with Office 365 and related resources hosted within Microsoft’s South-Central US datacenter. The service disruption was reported as fully resolved by Microsoft on Sept 7th at 1:40am PDT. Throughout the duration of the incident, Okta-provided services continued to function normally.\r\n\r\nMicrosoft reports a severe weather event, including lightning strikes, occurred near one of the South-Central US datacenters. This resulted in a power voltage increase which impacted cooling systems. Automated datacenter procedures designed to ensure data and hardware integrity went into effect and critical hardware entered a structured power down process.\r\n\r\nPlease refer to your [Office 365 Service Health](https://portal.office.com/adminportal/home#/servicehealth) dashboard for further details regarding this incident and mitigating steps.","Status__c":"Resolved","Start_Date__c":"2018-09-04","End_Date__c":"2018-09-04","Last_Updated__c":"2018-09-12T23:02:18.000+0000","LastModifiedDate":"2018-09-12T23:02:18.000+0000","Category__c":"Service Disruption","Size__c":"Large","Impacted_Cells__c":"okta-emea.com:1;okta.com:1;okta.com:2;okta.com:3;okta.com:4;okta.com:5;okta.com:6;okta.com:7;oktapreview.com:1;oktapreview.com:2","Impacted_Audience__c":"End user","Service_Feature__c":"Third Party","Start_Time__c":"2018-09-04T13:36:00.000+0000","CreatedDate":"2018-09-04T13:39:52.000+0000","CurrencyIsoCode":"USD"},{"attributes":{"type":"Incident__c","url":"/services/data/v48.0/sobjects/Incident__c/a9C0Z0000008PU1UAM"},"Id":"a9C0Z0000008PU1UAM","Duration__c":0,"Log__c":"Resolved: An issue impacting authentication and system responsiveness for a subset of end-users and administrators accessing Microsoft Office 365 has subsided. Further investigation of the issue found the errors were correlated with a service degradation reported by Microsoft. Microsoft reported this is related to mitigation activities form yesterday's service disruption (MO147606):\r\n\r\n\"We identified and reverted an update that was causing multiple services to be throttled. We're conducting tests to ensure that impact has been resolved. Further updates can be found in the admin portal under EX147785 and MO147789.\"","Status__c":"Resolved","Start_Date__c":"2018-09-05","End_Date__c":"2018-09-05","Last_Updated__c":"2018-09-05T19:31:17.000+0000","LastModifiedDate":"2018-09-05T19:31:17.000+0000","Category__c":"Service Disruption","Size__c":"Small","Impacted_Cells__c":"okta-emea.com:1;okta.com:1;okta.com:2;okta.com:3;okta.com:4;okta.com:5;okta.com:6;okta.com:7","Impacted_Audience__c":"Admin;End user","Service_Feature__c":"Third Party","Start_Time__c":"2018-09-05T18:01:00.000+0000","CreatedDate":"2018-09-05T18:01:55.000+0000","CurrencyIsoCode":"USD"},{"attributes":{"type":"Incident__c","url":"/services/data/v48.0/sobjects/Incident__c/a9C0Z0000004EAAUA2"},"Id":"a9C0Z0000004EAAUA2","Duration__c":26,"Log__c":"An issue impacting search functionality in US Preview Cell 1 has been resolved. A root cause analysis will be posted here within 2 business days.\r\n\r\n===============\r\n\r\nResolved: On July 5th, beginning at 10:55am PDT, Okta experienced a service degradation in US Preview Cell 1. The issue impacted customers who have enabled Early Access to User Search and are leveraging the /api/v1/users?search endpoint. The root cause was traced to an inconsistency in the deployment and validation choreography between our test and preview environments for the Search micro-service. This inconsistency prevented an incompatible configuration setting from being detected in the test environment, causing the Search endpoint to fail and return 503 error responses. The issue was resolved by reverting the change which caused the regression, allowing the Search micro-service to recover by 11:21am PDT. The inconsistency in the Search micro service's deployment and validation choreography has been resolved.\r\n<br><br> \r\nEvent Root Cause Analysis details can be found [here](https://support.okta.com/help/Documentation/Knowledge_Article/Root-Cause-Analysis-Service-Degradation-07-05-2018).","Status__c":"Resolved","Start_Date__c":"2018-07-05","End_Date__c":"2018-07-05","Last_Updated__c":"2018-07-09T05:50:19.000+0000","LastModifiedDate":"2018-07-09T05:50:19.000+0000","Category__c":"Service Degradation","Size__c":"Medium","Incident_Title__c":"Service Degradation - User Search API","Impacted_Cells__c":"oktapreview.com:1","Service_Feature__c":"Core Service","Start_Time__c":"2018-07-08T17:55:00.000+0000","CreatedDate":"2018-07-05T18:15:09.000+0000","CurrencyIsoCode":"USD"},{"attributes":{"type":"Incident__c","url":"/services/data/v48.0/sobjects/Incident__c/a9C0Z0000004EA5UAM"},"Id":"a9C0Z0000004EA5UAM","Duration__c":0,"Log__c":"Okta received reports from customers who were unable to reach our service. This appeared to be related to a broader Internet Service Provider connectivity issue with Comcast, as reported extensively on Twitter. We actively monitored the situation and confirmed all services were operational.\r\n\r\n---------\r\n\r\nResolved: On June 29th, beginning at 9:43am, Okta began receiving reports from some customers who were unable to access Okta. Upon immediate investigation, it was determined that Okta's services were operating normally. The root cause was a network disruption, in some geographic locations, with a major internet provider. Given the breadth of the 3rd party internet disruption, Okta proactively posted to https://trust.okta.com to inform customers that we are aware of the situation and that they may be impacted. Okta is continuing to closely monitor and ensure that Okta's services remain unaffected.","Status__c":"Resolved","Start_Date__c":"2018-06-29","End_Date__c":"2018-06-29","Last_Updated__c":"2018-06-29T20:20:19.000+0000","LastModifiedDate":"2018-06-29T20:20:19.000+0000","Category__c":"Service Degradation","Size__c":"Small","Impacted_Cells__c":"okta-emea.com:1;okta.com:1;okta.com:2;okta.com:3;okta.com:4;okta.com:5;okta.com:6;okta.com:7;oktapreview.com:1;oktapreview.com:2","Impacted_Audience__c":"Admin;API Products;End user","Service_Feature__c":"Third Party","Start_Time__c":"2018-06-29T17:36:00.000+0000","CreatedDate":"2018-06-29T17:39:59.000+0000","CurrencyIsoCode":"USD"},{"attributes":{"type":"Incident__c","url":"/services/data/v48.0/sobjects/Incident__c/a9C0Z0000004E5ZUAU"},"Id":"a9C0Z0000004E5ZUAU","Duration__c":0,"Log__c":"Resolved: On June 6th, beginning at 5:12am PDT, Okta experienced elevated CPU processing on application servers across all cells triggering automated alerts. Given the quantity of servers with increased CPU processing, customer impact was a possibility and hence the determination was made to post proactively to the [https://trust.okta.com](https://trust.okta.com) page. The increase in processing levels however was sustainable by the infrastructure and thus customer impact through the duration of the incident remained negligible. The increase in CPU processing levels ended at 5:59am PDT when an operational script, related to edge service protection, was terminated. The script has been updated to avoid a recurrence of the increase to CPU processing levels.\r\n<br><br>\r\nEvent Root Cause Analysis details can be found [here](\thttps://support.okta.com/help/Documentation/Knowledge_Article/Root-Cause-Analysis-Service-Alert-06-06-2018).","Status__c":"Resolved","Start_Date__c":"2018-06-07","Last_Updated__c":"2018-06-11T03:15:36.000+0000","LastModifiedDate":"2018-06-11T03:15:36.000+0000","Category__c":"Service Degradation","Size__c":"Small","Incident_Title__c":"Service Alert - Increased CPU utilization triggered monitoring alerts.","Impacted_Cells__c":"EU1;OK1;OK2;OK3;OK4;OK5;OK6;Preview","Service_Feature__c":"Core Service","CreatedDate":"2018-06-07T12:38:59.000+0000","CurrencyIsoCode":"USD"},{"attributes":{"type":"Incident__c","url":"/services/data/v48.0/sobjects/Incident__c/a9C0Z0000004E5UUAU"},"Id":"a9C0Z0000004E5UUAU","Duration__c":33,"Log__c":"An issue impacting network connectivity to US Cell 6 was experienced and has now been resolved. Root cause analysis will be posted here within 48 hours.\r\n\r\n=================================\r\n\r\nResolved: On May 31st, beginning at 12:13am PDT, Okta experienced a broad service disruption to US Cell 6. The disruption triggered Okta's automated monitoring, alerting the Engineering team who determined that both the application and its underlying cloud computing infrastructure was unreachable. This pointed to a multi-availability zone outage in Okta's cloud infrastructure and network provider. This was confirmed by our provider, and the decision was made to fail over to the cell's Disaster Recovery (DR) instance and the process was initiated. However before the DR failover process completed, network connectivity was restored and within approximately 5 minutes, automated system recovery processes had returned the cell to normal performance levels; ending the disruption at 12:46am PDT.\r\n\r\nEvent Root Cause Analysis details can be found [here](https://support.okta.com/help/Documentation/Knowledge_Article/Root-Cause-Analysis-Service-Disruption-05312018).","Status__c":"Resolved","Start_Date__c":"2018-05-31","End_Date__c":"2018-05-31","Last_Updated__c":"2018-06-06T23:38:38.000+0000","LastModifiedDate":"2018-06-06T23:38:38.000+0000","Category__c":"Service Disruption","Size__c":"Medium","Impacted_Cells__c":"okta.com:6","Impacted_Audience__c":"Admin;API Products;End user","Service_Feature__c":"Core Service","Start_Time__c":"2018-05-31T07:13:00.000+0000","CreatedDate":"2018-05-31T07:35:01.000+0000","CurrencyIsoCode":"USD"},{"attributes":{"type":"Incident__c","url":"/services/data/v48.0/sobjects/Incident__c/a9C0Z000000TNhVUAW"},"Id":"a9C0Z000000TNhVUAW","Duration__c":0,"Log__c":"Resolved: 1:40pm PDT. The issue impacting e-mail delivery to 'support@okta.com' and 'developer@okta.com' has been resolved.\r\n\r\n=======================\r\n\r\nCustomer Advisory: Okta is currently experiencing issues with receiving customer e-mails to 'support@okta.com' and 'developer@okta.com'. This does not impact the Okta service availability or any e-mails sent from your Okta tenant.\r\n\r\nIf you need assistance with a new or existing Okta customer support incident, please call Okta at (800) 219-0964. Additional international support numbers may be found at [Okta Support] (https://support.okta.com/help).","Status__c":"Resolved","Start_Date__c":"2018-05-22","Last_Updated__c":"2018-05-22T20:45:00.000+0000","LastModifiedDate":"2018-05-22T20:45:00.000+0000","Category__c":"Service Degradation","Size__c":"Small","Impacted_Audience__c":"Admin","Service_Feature__c":"Third Party","CreatedDate":"2018-05-22T17:54:46.000+0000","CurrencyIsoCode":"USD"},{"attributes":{"type":"Incident__c","url":"/services/data/v48.0/sobjects/Incident__c/a9C0Z000000TNgDUAW"},"Id":"a9C0Z000000TNgDUAW","Duration__c":51,"Log__c":"An issue impacting authentication and user interactive sessions in US Cell 6 is being investigated. End-Users and Admins may experience HTTP 500/503 errors while attempting to login. We are actively working to mitigate the issue and return the service to normal as quickly as possible\r\n\r\n--------------------------\r\n\r\nResolved: On May 17th, beginning at 11:28am PDT, Okta experienced a service disruption in US Cell 6. End-User and Administrators attempting to authenticate to their Okta tenant may have experienced HTTP 5xx errors. Through the duration of the incident, approximately 30% of US Cell 6 users attempting primary or multi-factor authentication encountered errors and would have needed to retry. Other requests, and End-Users and Administrators who already had an active session did not experience any impact.\r\n\r\nThe issue surfaced when the cell's session token management cluster became overwhelmed by a sudden increase in the number of active sessions. Automated monitoring failed to effectively alert before the cluster was nearing its maximum capacity.\r\n\r\nThe issue was resolved by 12:19pm PDT after operational actions were taken to manage the session count while capacity was added to the cluster. Immediately following the incident, Okta corrected the monitoring gap to prevent a future recurrence. Architectural design work is being initiated to better manage session counts and their expiry.\r\n\r\nEvent Root Cause Analysis details can be found [here](https://support.okta.com/help/Documentation/Knowledge_Article/Root-Cause-Analysis-Service-Disruption-05182018).","Status__c":"Resolved","Start_Date__c":"2018-05-17","End_Date__c":"2018-05-17","Last_Updated__c":"2018-05-22T02:03:34.000+0000","LastModifiedDate":"2018-05-22T02:03:34.000+0000","Category__c":"Service Disruption","Size__c":"Large","Impacted_Cells__c":"okta.com:6","Impacted_Audience__c":"Admin;End user","Service_Feature__c":"Core Service","Start_Time__c":"2018-05-17T18:28:00.000+0000","CreatedDate":"2018-05-17T18:49:34.000+0000","CurrencyIsoCode":"USD"},{"attributes":{"type":"Incident__c","url":"/services/data/v48.0/sobjects/Incident__c/a9C0Z000000TNfAUAW"},"Id":"a9C0Z000000TNfAUAW","Duration__c":42,"Log__c":"Resolved: Okta experienced a service degradation in US Cell 6 between 7:03pm and 7:45pm PDT due to a hardware failure and failover to secondary database infrastructure. Administrative functions like user creation, modification, and deletion may have experienced a period of [Read-Only mode] (https://support.okta.com/help/Documentation/Knowledge_Article/Read-only-Mode) during this time as corrective actions were taken.\r\n\r\nEvent Root Cause Analysis details can be found [here](https://support.okta.com/help/Documentation/Knowledge_Article/Root-Cause-Analysis-Service-Disruption-05062018).","Status__c":"Resolved","Start_Date__c":"2018-05-06","Last_Updated__c":"2018-05-10T05:55:41.000+0000","LastModifiedDate":"2018-05-10T05:55:41.000+0000","Category__c":"Service Degradation","Size__c":"Medium","Impacted_Cells__c":"OK6","Impacted_Audience__c":"Admin;API Products","Service_Feature__c":"Core Service","Start_Time__c":"2018-05-07T02:03:00.000+0000","CreatedDate":"2018-05-07T02:20:37.000+0000","CurrencyIsoCode":"USD"},{"attributes":{"type":"Incident__c","url":"/services/data/v48.0/sobjects/Incident__c/a9C0Z000000TNeqUAG"},"Id":"a9C0Z000000TNeqUAG","Duration__c":55,"Log__c":"Resolved: On April 24th, at 7:53am PDT, Okta experienced a broad service disruption to US Cell 2. End users in US Cell 2 attempting to reach Okta received HTTP 500 errors, and customer integrations attempting to reach Okta API endpoints in US Cell 2 encountered elevated response times and errors. Operational mitigation steps commenced immediately at 7:53am. The greatest frequency of errors was a 14 minute period between 7:53am and 8:07am. The success rate began to gradually improve from 8:07am, and by 8:25am approximately 80% of requests were successfully returned. The cell was fully restored by 8:48am. \r\n<br><br>\r\nRoot cause was traced to a combination of a procedural error and an automation bug while security patching infrastructure within US Cell 2. The procedural error targeted the security patching to a high usage cache cluster. The automation used to apply the patch disrupted the cluster’s ability to failover across nodes. \r\n<br><br>\r\nTo prevent recurrence of this type of issue in the future, Engineering has immediately implemented additional safeguards to review and confirm operational patching procedures. Additionally, Engineering has prioritized a deep technical assessment to strengthen the patching automation for cache clusters.\r\n\r\nEvent Root Cause Analysis details can be found [here](https://support.okta.com/help/Documentation/Knowledge_Article/Root-Cause-Analysis-Service-Disruption-04242018).","Status__c":"Resolved","Start_Date__c":"2018-04-24","End_Date__c":"2018-04-24","Last_Updated__c":"2018-05-01T22:59:56.000+0000","LastModifiedDate":"2018-05-01T22:59:56.000+0000","Category__c":"Service Disruption","Size__c":"Large","Impacted_Cells__c":"OK2","Impacted_Audience__c":"Admin;API Products;End user","Service_Feature__c":"Core Service","Start_Time__c":"2018-04-24T15:07:00.000+0000","CreatedDate":"2018-04-24T15:08:06.000+0000","CurrencyIsoCode":"USD"},{"attributes":{"type":"Incident__c","url":"/services/data/v48.0/sobjects/Incident__c/a9C0Z000000TNelUAG"},"Id":"a9C0Z000000TNelUAG","Duration__c":339,"Log__c":"On April 23, at 9:04am PDT, Okta initiated steps to rate limit the Events API endpoint across all preview and production cells so that a fix could be applied. While the fix was in progress, customer integrations attempting to reach the /api/v1/events endpoint would have received a 429 error response. By 2:43pm PDT, the fix had been fully applied across all cells.","Status__c":"Resolved","Start_Date__c":"2018-04-23","End_Date__c":"2018-04-23","Last_Updated__c":"2018-05-01T18:20:55.000+0000","LastModifiedDate":"2018-05-01T18:20:55.000+0000","Category__c":"Service Degradation","Size__c":"Large","Impacted_Cells__c":"EU1;OK1;OK2;OK3;OK4;OK5;OK6;Preview;Preview2","Impacted_Audience__c":"API Products","Service_Feature__c":"Core Service","Start_Time__c":"2018-04-23T16:18:00.000+0000","CreatedDate":"2018-04-23T16:18:41.000+0000","CurrencyIsoCode":"USD"},{"attributes":{"type":"Incident__c","url":"/services/data/v48.0/sobjects/Incident__c/a9C2A000000bmK1UAI"},"Id":"a9C2A000000bmK1UAI","Duration__c":323,"Log__c":"An issue impacting IDP-Initiated Office 365 Office Portal SSO for a subset of end-users in all cells was investigated with Microsoft. Customers may have seen page load failures for portal.microsoftonline.com. As a workaround, customers were able to utilize any of the app-specific chiclets to access O365, then navigate to the O365 Portal from that location.\r\n\r\nResolved: On March 30 at 4:22AM PDT, Okta experienced an issue impacting IDP-Initiated Office 365 Portal SSO for a subset of end-users, specifically those with IPV4 addresses. Users attempting to use the Office 365 Portal chiclet during the incident experienced page load failures. SP-Initiated SSO to the Office 365 Portal, as well as all other Okta Microsoft chiclets, continued to work normally throughout the incident. After it was determined that the issue appeared to be on the application vendor side, Okta immediately reported the matter and worked with Microsoft on potential workarounds and resolution. By 9:45 AM PDT, the problem had been resolved. Root cause analysis and next steps are provided in Microsoft incident MO133118 via the Microsoft Admin Center. Okta will be enhancing monitoring to detect similar Office 365 Portal issues faster in the future to be able to report them Microsoft as early as possible.","Status__c":"Resolved","Start_Date__c":"2018-03-30","Last_Updated__c":"2018-04-03T01:12:53.000+0000","LastModifiedDate":"2018-04-03T01:12:53.000+0000","Category__c":"Service Degradation","Size__c":"Large","Impacted_Cells__c":"EU1;OK1;OK2;OK3;OK4;OK5;OK6;Preview","Impacted_Audience__c":"End user","Service_Feature__c":"Third Party","CreatedDate":"2018-03-30T14:36:52.000+0000","CurrencyIsoCode":"USD"},{"attributes":{"type":"Incident__c","url":"/services/data/v48.0/sobjects/Incident__c/a9C2A0000004CSnUAM"},"Id":"a9C2A0000004CSnUAM","Duration__c":3300,"Log__c":"Okta is currently investigating an issue with Provisioning Setup and IDP-Initiated Single Sign-On functionality for the following Office 365 Apps: Yammer, CRM and Teams. SP-Initiated Single-Sign-On is not affected. Please contact support if you are having difficulty setting up provisioning and we will assist with implementation of an applicable workaround as we investigate further.","Status__c":"Resolved","Start_Date__c":"2018-03-20","End_Date__c":"2018-03-21","Last_Updated__c":"2018-03-22T21:43:38.000+0000","LastModifiedDate":"2018-03-22T21:43:38.000+0000","Category__c":"Feature Disruption","Size__c":"Large","Impacted_Cells__c":"EU1;OK1;OK2;OK3;OK4;OK5;OK6;Preview","Impacted_Audience__c":"Admin;End user","Service_Feature__c":"Core Service","Start_Time__c":"2018-03-20T22:15:00.000+0000","CreatedDate":"2018-03-20T22:11:19.000+0000","CurrencyIsoCode":"USD"},{"attributes":{"type":"Incident__c","url":"/services/data/v48.0/sobjects/Incident__c/a9C2A0000004CSTUA2"},"Id":"a9C2A0000004CSTUA2","Duration__c":121,"Log__c":"Okta is currently experiencing an issue in US Cell 6. Administrators will not be able to see up to date user / group information within their Okta tenant. No authentication impact for existing users. Our TechOps and Engineering teams are actively investigating this issue.","Status__c":"Resolved","Start_Date__c":"2018-03-16","End_Date__c":"2018-03-16","Last_Updated__c":"2018-03-21T14:14:35.000+0000","LastModifiedDate":"2018-03-21T14:14:35.000+0000","Category__c":"Service Degradation","Size__c":"Large","Impacted_Cells__c":"OK6","Impacted_Audience__c":"Admin","Service_Feature__c":"Core Service","Start_Time__c":"2018-03-16T17:11:00.000+0000","CreatedDate":"2018-03-16T17:12:33.000+0000","CurrencyIsoCode":"USD"},{"attributes":{"type":"Incident__c","url":"/services/data/v48.0/sobjects/Incident__c/a9C2A000000PBikUAG"},"Id":"a9C2A000000PBikUAG","Duration__c":352,"Log__c":"Okta is currently experiencing an issue with the API Access Management feature in our US Preview Cell. End-users attempting to access endpoints for this feature may receive an HTTP 500 error. Okta is currently working to resolve the issue and we will post an update shortly.","Status__c":"Resolved","Start_Date__c":"2018-03-07","Last_Updated__c":"2018-03-10T00:16:29.000+0000","LastModifiedDate":"2018-03-10T00:16:29.000+0000","Category__c":"Service Disruption","Size__c":"Large","Impacted_Cells__c":"oktapreview.com:1","Impacted_Audience__c":"End user","Service_Feature__c":"Core Service","CreatedDate":"2018-03-07T18:00:08.000+0000","CurrencyIsoCode":"USD"},{"attributes":{"type":"Incident__c","url":"/services/data/v48.0/sobjects/Incident__c/a9C2A000000PBipUAG"},"Id":"a9C2A000000PBipUAG","Duration__c":53,"Log__c":"Okta is currently experiencing an issue in US Cell 6 whereby users attempting to login to Okta are experiencing slowness, timeouts, or HTTP 500 errors when attempting to login or navigate their Okta tenant.","Status__c":"Resolved","Start_Date__c":"2018-03-07","Last_Updated__c":"2018-03-09T23:40:34.000+0000","LastModifiedDate":"2018-03-09T23:40:34.000+0000","Category__c":"Service Disruption","Size__c":"Large","Impacted_Cells__c":"okta.com:6","Impacted_Audience__c":"Admin;API Products;End user","Service_Feature__c":"Core Service","CreatedDate":"2018-03-08T01:24:58.000+0000","CurrencyIsoCode":"USD"},{"attributes":{"type":"Incident__c","url":"/services/data/v48.0/sobjects/Incident__c/a9C2A000000PBiBUAW"},"Id":"a9C2A000000PBiBUAW","Duration__c":57,"Log__c":"On February 25th, Okta experienced an issue in US Preview cell beginning at 9:00am PST which resulted in approximately 2.3% of interactive requests for end-users and administrators receiving HTTP 500 or 504 errors. Okta Engineering's investigation found that an outbound internet gateway high availability solution for one of the three availability zones in the US Preview cell had failed. Following a manual fail-over, the issue was mitigated by 9:33am for all services except Syslog v2 which was fully restored by 9:57am PST. To mitigate future impact, improvements are planned for the high availability infrastructure which manages outbound internet traffic.","Status__c":"Resolved","Start_Date__c":"2018-02-25","End_Date__c":"2018-02-25","Last_Updated__c":"2018-02-28T06:38:35.000+0000","LastModifiedDate":"2018-02-28T06:38:35.000+0000","Category__c":"Service Disruption","Size__c":"Large","Impacted_Cells__c":"oktapreview.com:1","Impacted_Audience__c":"Admin;End user","Service_Feature__c":"Core Service","Start_Time__c":"2018-02-25T17:30:00.000+0000","CreatedDate":"2018-02-25T17:30:36.000+0000","CurrencyIsoCode":"USD"}]Okta Status Site Guest UserGuestokta-emea.com:1,okta.com:1,okta.com:2,okta.com:3,okta.com:4,okta.com:5,okta.com:6,okta.com:7,oktapreview.com:1,oktapreview.com:2,okta.com:8,okta.com:9,okta.com:10,okta.com:11[{"year":2010, "uptime":99.98, "month":["100.000","100.000","100.000","100.000","99.955","100.000","100.000","99.933","99.928","99.960","100.000","100.000"]},{"year":2011, "uptime":99.96, "month":["99.973","99.980","100.000","99.977","99.969","99.954","100.000","99.917","99.889","99.964","100.000","99.953"]},{"year":2012, "uptime":99.9966, "month":["100.000","100.000","99.996","100.000","100.000","100.000","100.000","99.964","100.000","100.000","100.000","100.000"]},{"year":2013, "uptime":99.9994, "month":["99.993","100.000","100.000","100.000","100.000","100.000","100.000","100.000","100.000","100.000","100.000","100.000"]},{"year":2014, "uptime":99.95, "month":["100.000","100.000","100.000","100.000","100.000","100.000","100.000","99.610","99.824","99.993","100.000","99.980"]},{"year":2015, "uptime":100.0000, "month":["100.000","100.000","100.000","100.000","100.000","100.000","100.000","100.000","100.000","100.000","100.000","100.000"]},{"year":2016, "uptime":99.95, "month":["100.000","100.000","100.000","100.000","100.000","100.000","100.000","100.000","100.000","99.384","100.000","100.000"]},{"year":2017, "uptime":99.97, "month":["99.969","99.692","100.000","100.000","100.000","100.000","100.000","100.000","100.000","100.000","100.000","100.000"]},{"year":2018, "uptime":100.0000, "month":["100.000","100.000","100.000","100.000","100.000","100.000","100.000","100.000","100.000","100.000","100.000","100.000"]},{"year":2019, "uptime":100.0000, "month":["100.000","100.000","100.000","100.000","100.000","100.000","100.000","100.000","100.000","100.000","100.000","100.000"]},{"year":2020, "uptime":100.0000, "month":["100.000","100.000","","","","","","","","","",""]}][{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0171Y0000rEQNXBQA5"},"Id":"0171Y0000rEQNXBQA5","ParentId":"a9C1Y0000004DyLUAU","CreatedDate":"2019-09-30T22:50:16.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0170Z0000b3ZRCIQA4"},"Id":"0170Z0000b3ZRCIQA4","ParentId":"a9C0Z000000TNeqUAG","CreatedDate":"2018-04-24T15:08:06.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0170Z0000bWVjUiQAL"},"Id":"0170Z0000bWVjUiQAL","ParentId":"a9C0Z000000TNgDUAW","CreatedDate":"2018-05-17T18:49:34.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0170Z0000bdOClFQAW"},"Id":"0170Z0000bdOClFQAW","ParentId":"a9C0Z000000TNhVUAW","CreatedDate":"2018-05-22T17:54:46.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0170Z0000b2IZ3NQAW"},"Id":"0170Z0000b2IZ3NQAW","ParentId":"a9C0Z000000TNelUAG","CreatedDate":"2018-04-23T16:18:41.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0170Z0000bH0XiBQAV"},"Id":"0170Z0000bH0XiBQAV","ParentId":"a9C0Z000000TNfAUAW","CreatedDate":"2018-05-07T02:20:37.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0171Y0000sCTiwVQAT"},"Id":"0171Y0000sCTiwVQAT","ParentId":"a9C1Y0000004E2wUAE","CreatedDate":"2019-11-06T17:35:16.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0170Z0000byaXfYQAU"},"Id":"0170Z0000byaXfYQAU","ParentId":"a9C0Z0000004E5ZUAU","CreatedDate":"2018-06-07T12:38:59.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0170Z0000eAYE98QAH"},"Id":"0170Z0000eAYE98QAH","ParentId":"a9C0Z0000008PUVUA2","CreatedDate":"2018-09-18T00:58:30.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0172A0000e3AmZOQA0"},"Id":"0172A0000e3AmZOQA0","ParentId":"a9C2A0000004CSTUA2","CreatedDate":"2018-03-16T17:12:34.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000N7c11cIQA"},"Id":"017F00000N7c11cIQA","ParentId":"a9CF0000000XZIGMA4","CreatedDate":"2015-09-16T21:19:06.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0172A0000XAa1hOQQR"},"Id":"0172A0000XAa1hOQQR","ParentId":"a9C2A000000PBZNUA4","CreatedDate":"2016-09-28T19:56:21.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0172A0000XE7EB3QQN"},"Id":"0172A0000XE7EB3QQN","ParentId":"a9C2A000000PBZwUAO","CreatedDate":"2016-10-04T18:05:36.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0171Y0000k28E8gQAE"},"Id":"0171Y0000k28E8gQAE","ParentId":"a9C1Y000000GpJuUAK","CreatedDate":"2018-12-13T13:04:33.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0172A0000YcwVFkQQM"},"Id":"0172A0000YcwVFkQQM","ParentId":"a9C2A000000XZfGUAW","CreatedDate":"2017-02-02T12:57:47.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0172A0000XmAKzzQQG"},"Id":"0172A0000XmAKzzQQG","ParentId":"a9C2A000000XZXbUAO","CreatedDate":"2016-11-15T20:58:57.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0172A0000XuJp7uQQC"},"Id":"0172A0000XuJp7uQQC","ParentId":"a9C2A000000XZYPUA4","CreatedDate":"2016-11-30T19:10:08.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0172A0000XanMc9QQE"},"Id":"0172A0000XanMc9QQE","ParentId":"a9C2A000000XZWTUA4","CreatedDate":"2016-10-27T23:19:41.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0172A0000YktEugQQE"},"Id":"0172A0000YktEugQQE","ParentId":"a9C2A000000XZiUUAW","CreatedDate":"2017-02-15T16:29:10.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0170Z0000drgXEkQAM"},"Id":"0170Z0000drgXEkQAM","ParentId":"a9C0Z0000008PU1UAM","CreatedDate":"2018-09-05T18:01:55.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0170Z0000bpMJXqQAO"},"Id":"0170Z0000bpMJXqQAO","ParentId":"a9C0Z0000004E5UUAU","CreatedDate":"2018-05-31T07:35:02.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0172A0000e6vxu3QAA"},"Id":"0172A0000e6vxu3QAA","ParentId":"a9C2A0000004CSnUAM","CreatedDate":"2018-03-20T22:11:19.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0172A0000ayVnJsQAK"},"Id":"0172A0000ayVnJsQAK","ParentId":"a9C2A000000bmESUAY","CreatedDate":"2017-08-04T12:52:51.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0172A0000b96tesQAA"},"Id":"0172A0000b96tesQAA","ParentId":"a9C2A000000bmJDUAY","CreatedDate":"2017-08-21T17:36:30.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0172A0000axTPnpQAG"},"Id":"0172A0000axTPnpQAG","ParentId":"a9C2A000000bmDeUAI","CreatedDate":"2017-08-03T12:31:20.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0172A0000bGmbCSQAZ"},"Id":"0172A0000bGmbCSQAZ","ParentId":"a9C2A000000Kyv2UAC","CreatedDate":"2017-08-31T17:37:43.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0172A0000b7Rg1CQAS"},"Id":"0172A0000b7Rg1CQAS","ParentId":"a9C2A000000bmHvUAI","CreatedDate":"2017-08-17T23:04:30.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0172A0000YBiCLJQQ3"},"Id":"0172A0000YBiCLJQQ3","ParentId":"a9C2A000000XZclUAG","CreatedDate":"2016-12-21T21:56:39.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0172A0000YBh4fAQQR"},"Id":"0172A0000YBh4fAQQR","ParentId":"a9C2A000000XZcgUAG","CreatedDate":"2016-12-21T17:20:30.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0172A0000XjOQXhQQO"},"Id":"0172A0000XjOQXhQQO","ParentId":"a9C2A000000XZX2UAO","CreatedDate":"2016-11-10T19:09:55.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0172A0000XuGYxRQQW"},"Id":"0172A0000XuGYxRQQW","ParentId":"a9C2A000000XZYKUA4","CreatedDate":"2016-11-30T01:02:28.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0170Z0000dq55bAQAQ"},"Id":"0170Z0000dq55bAQAQ","ParentId":"a9C0Z0000008PTrUAM","CreatedDate":"2018-09-04T13:39:52.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0170Z0000dq7TZKQA2"},"Id":"0170Z0000dq7TZKQA2","ParentId":"a9C0Z0000008PTwUAM","CreatedDate":"2018-09-04T21:57:16.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0172A0000YS5CkxQQF"},"Id":"0172A0000YS5CkxQQF","ParentId":"a9C2A000000XZeNUAW","CreatedDate":"2017-01-17T19:34:18.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0172A0000Y5q6tBQQQ"},"Id":"0172A0000Y5q6tBQQQ","ParentId":"a9C2A000000XZb9UAG","CreatedDate":"2016-12-16T14:50:00.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0172A0000ZtrmGjQQI"},"Id":"0172A0000ZtrmGjQQI","ParentId":"a9C2A000000blruUAA","CreatedDate":"2017-05-15T13:19:44.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000SifR7IIQU"},"Id":"017F00000SifR7IIQU","ParentId":"a9CF00000008OihMAE","CreatedDate":"2016-04-23T00:36:14.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0172A0000YyEfV3QQK"},"Id":"0172A0000YyEfV3QQK","ParentId":"a9C2A000000XZkGUAW","CreatedDate":"2017-03-02T22:16:34.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0172A0000ZmpbvtQQA"},"Id":"0172A0000ZmpbvtQQA","ParentId":"a9C2A000000blcLUAQ","CreatedDate":"2017-05-05T17:48:13.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0172A0000ZmpUXIQQ2"},"Id":"0172A0000ZmpUXIQQ2","ParentId":"a9C2A000000blcGUAQ","CreatedDate":"2017-05-05T17:07:11.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000KmSvSkIQK"},"Id":"017F00000KmSvSkIQK","ParentId":"a9CF0000000PBJAMA4","CreatedDate":"2015-06-11T01:25:08.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0172A0000aHNE4gQAH"},"Id":"0172A0000aHNE4gQAH","ParentId":"a9C2A000000bm9IUAQ","CreatedDate":"2017-06-20T04:06:05.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000V8IuOqIQK"},"Id":"017F00000V8IuOqIQK","ParentId":"a9CF00000008P5MMAU","CreatedDate":"2016-07-18T15:38:50.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000UcibJsIQI"},"Id":"017F00000UcibJsIQI","ParentId":"a9CF00000008P3VMAU","CreatedDate":"2016-07-01T09:50:21.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000K5dqXMIQY"},"Id":"017F00000K5dqXMIQY","ParentId":"a9CF0000000PBFNMA4","CreatedDate":"2015-05-12T11:55:12.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JbZMPBIQ4"},"Id":"017F00000JbZMPBIQ4","ParentId":"a9CF0000000PBBaMAO","CreatedDate":"2015-04-22T15:30:23.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQA7u8IQD"},"Id":"017F00000JQA7u8IQD","ParentId":"a9CF0000000PB8gMAG","CreatedDate":"2015-04-14T05:39:20.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yNnIQJ"},"Id":"017F00000JQ9yNnIQJ","ParentId":"a9CF0000000PB7GMAW","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yNoIQJ"},"Id":"017F00000JQ9yNoIQJ","ParentId":"a9CF0000000PB7HMAW","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yNpIQJ"},"Id":"017F00000JQ9yNpIQJ","ParentId":"a9CF0000000PB7IMAW","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yNqIQJ"},"Id":"017F00000JQ9yNqIQJ","ParentId":"a9CF0000000PB7JMAW","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yNrIQJ"},"Id":"017F00000JQ9yNrIQJ","ParentId":"a9CF0000000PB7KMAW","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yNsIQJ"},"Id":"017F00000JQ9yNsIQJ","ParentId":"a9CF0000000PB7LMAW","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yNtIQJ"},"Id":"017F00000JQ9yNtIQJ","ParentId":"a9CF0000000PB7MMAW","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yNuIQJ"},"Id":"017F00000JQ9yNuIQJ","ParentId":"a9CF0000000PB7NMAW","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yNvIQJ"},"Id":"017F00000JQ9yNvIQJ","ParentId":"a9CF0000000PB7OMAW","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yNwIQJ"},"Id":"017F00000JQ9yNwIQJ","ParentId":"a9CF0000000PB7PMAW","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yNxIQJ"},"Id":"017F00000JQ9yNxIQJ","ParentId":"a9CF0000000PB7QMAW","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yNyIQJ"},"Id":"017F00000JQ9yNyIQJ","ParentId":"a9CF0000000PB7RMAW","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yNzIQJ"},"Id":"017F00000JQ9yNzIQJ","ParentId":"a9CF0000000PB7SMAW","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yO0IQJ"},"Id":"017F00000JQ9yO0IQJ","ParentId":"a9CF0000000PB7TMAW","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yO1IQJ"},"Id":"017F00000JQ9yO1IQJ","ParentId":"a9CF0000000PB7UMAW","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yO2IQJ"},"Id":"017F00000JQ9yO2IQJ","ParentId":"a9CF0000000PB7VMAW","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yO3IQJ"},"Id":"017F00000JQ9yO3IQJ","ParentId":"a9CF0000000PB7WMAW","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yO4IQJ"},"Id":"017F00000JQ9yO4IQJ","ParentId":"a9CF0000000PB7XMAW","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yO5IQJ"},"Id":"017F00000JQ9yO5IQJ","ParentId":"a9CF0000000PB7YMAW","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yO6IQJ"},"Id":"017F00000JQ9yO6IQJ","ParentId":"a9CF0000000PB7ZMAW","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yO7IQJ"},"Id":"017F00000JQ9yO7IQJ","ParentId":"a9CF0000000PB7aMAG","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yO8IQJ"},"Id":"017F00000JQ9yO8IQJ","ParentId":"a9CF0000000PB7bMAG","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yO9IQJ"},"Id":"017F00000JQ9yO9IQJ","ParentId":"a9CF0000000PB7cMAG","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yOAIQZ"},"Id":"017F00000JQ9yOAIQZ","ParentId":"a9CF0000000PB7dMAG","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yOBIQZ"},"Id":"017F00000JQ9yOBIQZ","ParentId":"a9CF0000000PB7eMAG","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yOCIQZ"},"Id":"017F00000JQ9yOCIQZ","ParentId":"a9CF0000000PB7fMAG","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yODIQZ"},"Id":"017F00000JQ9yODIQZ","ParentId":"a9CF0000000PB7gMAG","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yOEIQZ"},"Id":"017F00000JQ9yOEIQZ","ParentId":"a9CF0000000PB7hMAG","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yOFIQZ"},"Id":"017F00000JQ9yOFIQZ","ParentId":"a9CF0000000PB7iMAG","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yOGIQZ"},"Id":"017F00000JQ9yOGIQZ","ParentId":"a9CF0000000PB7jMAG","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yOHIQZ"},"Id":"017F00000JQ9yOHIQZ","ParentId":"a9CF0000000PB7kMAG","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yOIIQZ"},"Id":"017F00000JQ9yOIIQZ","ParentId":"a9CF0000000PB7lMAG","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yOJIQZ"},"Id":"017F00000JQ9yOJIQZ","ParentId":"a9CF0000000PB7mMAG","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yOKIQZ"},"Id":"017F00000JQ9yOKIQZ","ParentId":"a9CF0000000PB7nMAG","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yOLIQZ"},"Id":"017F00000JQ9yOLIQZ","ParentId":"a9CF0000000PB7oMAG","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yOMIQZ"},"Id":"017F00000JQ9yOMIQZ","ParentId":"a9CF0000000PB7pMAG","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yONIQZ"},"Id":"017F00000JQ9yONIQZ","ParentId":"a9CF0000000PB7qMAG","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yOOIQZ"},"Id":"017F00000JQ9yOOIQZ","ParentId":"a9CF0000000PB7rMAG","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yOPIQZ"},"Id":"017F00000JQ9yOPIQZ","ParentId":"a9CF0000000PB7sMAG","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yOQIQZ"},"Id":"017F00000JQ9yOQIQZ","ParentId":"a9CF0000000PB7tMAG","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yORIQZ"},"Id":"017F00000JQ9yORIQZ","ParentId":"a9CF0000000PB7uMAG","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yOSIQZ"},"Id":"017F00000JQ9yOSIQZ","ParentId":"a9CF0000000PB7vMAG","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yOTIQZ"},"Id":"017F00000JQ9yOTIQZ","ParentId":"a9CF0000000PB7wMAG","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yOUIQZ"},"Id":"017F00000JQ9yOUIQZ","ParentId":"a9CF0000000PB7xMAG","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yOVIQZ"},"Id":"017F00000JQ9yOVIQZ","ParentId":"a9CF0000000PB7yMAG","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yOWIQZ"},"Id":"017F00000JQ9yOWIQZ","ParentId":"a9CF0000000PB7zMAG","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yOXIQZ"},"Id":"017F00000JQ9yOXIQZ","ParentId":"a9CF0000000PB80MAG","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yOYIQZ"},"Id":"017F00000JQ9yOYIQZ","ParentId":"a9CF0000000PB81MAG","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yOZIQZ"},"Id":"017F00000JQ9yOZIQZ","ParentId":"a9CF0000000PB82MAG","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yOaIQJ"},"Id":"017F00000JQ9yOaIQJ","ParentId":"a9CF0000000PB83MAG","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yObIQJ"},"Id":"017F00000JQ9yObIQJ","ParentId":"a9CF0000000PB84MAG","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yOcIQJ"},"Id":"017F00000JQ9yOcIQJ","ParentId":"a9CF0000000PB85MAG","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yOdIQJ"},"Id":"017F00000JQ9yOdIQJ","ParentId":"a9CF0000000PB86MAG","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yOeIQJ"},"Id":"017F00000JQ9yOeIQJ","ParentId":"a9CF0000000PB87MAG","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yOfIQJ"},"Id":"017F00000JQ9yOfIQJ","ParentId":"a9CF0000000PB88MAG","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yOgIQJ"},"Id":"017F00000JQ9yOgIQJ","ParentId":"a9CF0000000PB89MAG","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yOhIQJ"},"Id":"017F00000JQ9yOhIQJ","ParentId":"a9CF0000000PB8AMAW","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yOiIQJ"},"Id":"017F00000JQ9yOiIQJ","ParentId":"a9CF0000000PB8BMAW","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yOjIQJ"},"Id":"017F00000JQ9yOjIQJ","ParentId":"a9CF0000000PB8CMAW","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yOkIQJ"},"Id":"017F00000JQ9yOkIQJ","ParentId":"a9CF0000000PB8DMAW","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yOlIQJ"},"Id":"017F00000JQ9yOlIQJ","ParentId":"a9CF0000000PB8EMAW","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yOmIQJ"},"Id":"017F00000JQ9yOmIQJ","ParentId":"a9CF0000000PB8FMAW","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yOnIQJ"},"Id":"017F00000JQ9yOnIQJ","ParentId":"a9CF0000000PB8GMAW","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yOoIQJ"},"Id":"017F00000JQ9yOoIQJ","ParentId":"a9CF0000000PB8HMAW","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yOpIQJ"},"Id":"017F00000JQ9yOpIQJ","ParentId":"a9CF0000000PB8IMAW","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yOqIQJ"},"Id":"017F00000JQ9yOqIQJ","ParentId":"a9CF0000000PB8JMAW","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yOrIQJ"},"Id":"017F00000JQ9yOrIQJ","ParentId":"a9CF0000000PB8KMAW","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yOsIQJ"},"Id":"017F00000JQ9yOsIQJ","ParentId":"a9CF0000000PB8LMAW","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yOtIQJ"},"Id":"017F00000JQ9yOtIQJ","ParentId":"a9CF0000000PB8MMAW","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yOuIQJ"},"Id":"017F00000JQ9yOuIQJ","ParentId":"a9CF0000000PB8NMAW","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yOvIQJ"},"Id":"017F00000JQ9yOvIQJ","ParentId":"a9CF0000000PB8OMAW","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yOwIQJ"},"Id":"017F00000JQ9yOwIQJ","ParentId":"a9CF0000000PB8PMAW","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yOxIQJ"},"Id":"017F00000JQ9yOxIQJ","ParentId":"a9CF0000000PB8QMAW","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yOyIQJ"},"Id":"017F00000JQ9yOyIQJ","ParentId":"a9CF0000000PB8RMAW","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yOzIQJ"},"Id":"017F00000JQ9yOzIQJ","ParentId":"a9CF0000000PB8SMAW","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yP0IQJ"},"Id":"017F00000JQ9yP0IQJ","ParentId":"a9CF0000000PB8TMAW","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yP1IQJ"},"Id":"017F00000JQ9yP1IQJ","ParentId":"a9CF0000000PB8UMAW","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yP2IQJ"},"Id":"017F00000JQ9yP2IQJ","ParentId":"a9CF0000000PB8VMAW","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yP3IQJ"},"Id":"017F00000JQ9yP3IQJ","ParentId":"a9CF0000000PB8WMAW","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yP4IQJ"},"Id":"017F00000JQ9yP4IQJ","ParentId":"a9CF0000000PB8XMAW","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yP5IQJ"},"Id":"017F00000JQ9yP5IQJ","ParentId":"a9CF0000000PB8YMAW","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yP6IQJ"},"Id":"017F00000JQ9yP6IQJ","ParentId":"a9CF0000000PB8ZMAW","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yP7IQJ"},"Id":"017F00000JQ9yP7IQJ","ParentId":"a9CF0000000PB8aMAG","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yP8IQJ"},"Id":"017F00000JQ9yP8IQJ","ParentId":"a9CF0000000PB8bMAG","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yP9IQJ"},"Id":"017F00000JQ9yP9IQJ","ParentId":"a9CF0000000PB8cMAG","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yPAIQZ"},"Id":"017F00000JQ9yPAIQZ","ParentId":"a9CF0000000PB8dMAG","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yPBIQZ"},"Id":"017F00000JQ9yPBIQZ","ParentId":"a9CF0000000PB8eMAG","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yPCIQZ"},"Id":"017F00000JQ9yPCIQZ","ParentId":"a9CF0000000PB8fMAG","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yHYIQZ"},"Id":"017F00000JQ9yHYIQZ","ParentId":"a9CF0000000PB78MAG","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yHZIQZ"},"Id":"017F00000JQ9yHZIQZ","ParentId":"a9CF0000000PB79MAG","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yHaIQJ"},"Id":"017F00000JQ9yHaIQJ","ParentId":"a9CF0000000PB7AMAW","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yNiIQJ"},"Id":"017F00000JQ9yNiIQJ","ParentId":"a9CF0000000PB7BMAW","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yNjIQJ"},"Id":"017F00000JQ9yNjIQJ","ParentId":"a9CF0000000PB7CMAW","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yNkIQJ"},"Id":"017F00000JQ9yNkIQJ","ParentId":"a9CF0000000PB7DMAW","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JQ9yNlIQJ"},"Id":"017F00000JQ9yNlIQJ","ParentId":"a9CF0000000PB7EMAW","CreatedDate":"2015-04-14T05:29:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0170Z0000cYSj9RQAT"},"Id":"0170Z0000cYSj9RQAT","ParentId":"a9C0Z0000004EAAUA2","CreatedDate":"2018-07-05T18:15:09.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000PhcK6DIQU"},"Id":"017F00000PhcK6DIQU","ParentId":"a9CF00000008OM2MAM","CreatedDate":"2016-01-04T18:09:55.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000PmIKP9IQO"},"Id":"017F00000PmIKP9IQO","ParentId":"a9CF00000008OMlMAM","CreatedDate":"2016-01-07T03:43:54.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000NTxzuSIQR"},"Id":"017F00000NTxzuSIQR","ParentId":"a9CF0000000XZKbMAO","CreatedDate":"2015-10-01T12:19:58.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0172A0000bZO71eQAD"},"Id":"0172A0000bZO71eQAD","ParentId":"a9C2A000000KyxrUAC","CreatedDate":"2017-09-26T13:16:54.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0172A0000bv1PXsQAM"},"Id":"0172A0000bv1PXsQAM","ParentId":"a9C2A000000Kz0CUAS","CreatedDate":"2017-10-19T17:04:56.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0172A0000cDinMDQAZ"},"Id":"0172A0000cDinMDQAZ","ParentId":"a9C2A000000Kz15UAC","CreatedDate":"2017-11-10T23:54:33.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0172A0000bPuQwbQAF"},"Id":"0172A0000bPuQwbQAF","ParentId":"a9C2A000000KyvbUAC","CreatedDate":"2017-09-13T14:33:13.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0172A0000XGw8wXQQR"},"Id":"0172A0000XGw8wXQQR","ParentId":"a9C2A000000PBaBUAW","CreatedDate":"2016-10-10T16:49:59.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0172A0000YbiwOrQQI"},"Id":"0172A0000YbiwOrQQI","ParentId":"a9C2A000000XZewUAG","CreatedDate":"2017-01-31T21:03:34.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0172A0000X2AyT4QQK"},"Id":"0172A0000X2AyT4QQK","ParentId":"a9C2A000000PBYUUA4","CreatedDate":"2016-09-20T18:52:14.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0172A0000X3Nwe4QQC"},"Id":"0172A0000X3Nwe4QQC","ParentId":"a9C2A000000PBYZUA4","CreatedDate":"2016-09-21T21:25:54.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0170Z0000fcb9sbQAA"},"Id":"0170Z0000fcb9sbQAA","ParentId":"a9C0Z0000008PW7UAM","CreatedDate":"2018-11-11T23:41:45.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0172A0000X3mlCnQQI"},"Id":"0172A0000X3mlCnQQI","ParentId":"a9C2A000000PBYeUAO","CreatedDate":"2016-09-22T13:12:00.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0172A0000eIqo6FQAR"},"Id":"0172A0000eIqo6FQAR","ParentId":"a9C2A000000bmK1UAI","CreatedDate":"2018-03-30T14:36:53.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0172A0000YD9IfHQQV"},"Id":"0172A0000YD9IfHQQV","ParentId":"a9C2A000000XZdZUAW","CreatedDate":"2016-12-23T17:34:39.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0172A0000XWliV8QQJ"},"Id":"0172A0000XWliV8QQJ","ParentId":"a9C2A000000XZVuUAO","CreatedDate":"2016-10-21T12:32:05.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0172A0000YMaIhNQQV"},"Id":"0172A0000YMaIhNQQV","ParentId":"a9C2A000000XZdeUAG","CreatedDate":"2017-01-09T16:12:20.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0172A0000YMaKHdQQN"},"Id":"0172A0000YMaKHdQQN","ParentId":"a9C2A000000XZdjUAG","CreatedDate":"2017-01-09T16:18:13.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0172A0000YOaZuLQQV"},"Id":"0172A0000YOaZuLQQV","ParentId":"a9C2A000000XZe3UAG","CreatedDate":"2017-01-11T17:10:13.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0172A0000ZY6QjkQQF"},"Id":"0172A0000ZY6QjkQQF","ParentId":"a9C2A000000XZsoUAG","CreatedDate":"2017-04-17T18:16:15.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0172A0000Zv44fQQQQ"},"Id":"0172A0000Zv44fQQQQ","ParentId":"a9C2A000000blrzUAA","CreatedDate":"2017-05-16T23:38:27.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000LIDzaKIQT"},"Id":"017F00000LIDzaKIQT","ParentId":"a9CF0000000PBLaMAO","CreatedDate":"2015-07-01T19:14:26.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000LFUUgFIQX"},"Id":"017F00000LFUUgFIQX","ParentId":"a9CF0000000PBLVMA4","CreatedDate":"2015-06-30T03:16:11.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000RQZEn4IQH"},"Id":"017F00000RQZEn4IQH","ParentId":"a9CF00000008OZQMA2","CreatedDate":"2016-03-09T23:30:15.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0172A0000ZBIEbYQQX"},"Id":"0172A0000ZBIEbYQQX","ParentId":"a9C2A000000XZl4UAG","CreatedDate":"2017-03-21T03:00:43.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0172A0000aDMIbhQAH"},"Id":"0172A0000aDMIbhQAH","ParentId":"a9C2A000000bm6BUAQ","CreatedDate":"2017-06-14T03:15:51.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0172A0000aMdrFpQAJ"},"Id":"0172A0000aMdrFpQAJ","ParentId":"a9C2A000000bmApUAI","CreatedDate":"2017-06-26T22:24:46.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0172A0000ahD4bJQAS"},"Id":"0172A0000ahD4bJQAS","ParentId":"a9C2A000000bmBsUAI","CreatedDate":"2017-07-11T13:24:32.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000Trktn8IQA"},"Id":"017F00000Trktn8IQA","ParentId":"a9CF00000008OvqMAE","CreatedDate":"2016-06-06T18:30:27.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000VRWdLBIQ1"},"Id":"017F00000VRWdLBIQ1","ParentId":"a9CF00000008P6PMAU","CreatedDate":"2016-07-28T23:59:57.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000Trj366IQA"},"Id":"017F00000Trj366IQA","ParentId":"a9CF00000008OvlMAE","CreatedDate":"2016-06-06T13:06:18.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0172A0000duuLFgQAM"},"Id":"0172A0000duuLFgQAM","ParentId":"a9C2A000000PBipUAG","CreatedDate":"2018-03-08T01:24:58.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0170Z0000cRTRSJQA5"},"Id":"0170Z0000cRTRSJQA5","ParentId":"a9C0Z0000004EA5UAM","CreatedDate":"2018-06-29T17:40:00.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000PCKAgiIQH"},"Id":"017F00000PCKAgiIQH","ParentId":"a9CF0000000KyuYMAS","CreatedDate":"2015-12-13T05:50:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000O1YFpfIQG"},"Id":"017F00000O1YFpfIQG","ParentId":"a9CF0000000blP2MAI","CreatedDate":"2015-10-23T19:05:05.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000OKYxvuIQD"},"Id":"017F00000OKYxvuIQD","ParentId":"a9CF0000000blT4MAI","CreatedDate":"2015-11-05T23:13:45.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0172A0000bzbOt9QAE"},"Id":"0172A0000bzbOt9QAE","ParentId":"a9C2A000000Kz0WUAS","CreatedDate":"2017-10-25T13:24:12.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000MKCXnCIQX"},"Id":"017F00000MKCXnCIQX","ParentId":"a9CF0000000PBToMAO","CreatedDate":"2015-08-14T20:32:12.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000MKCZuvIQH"},"Id":"017F00000MKCZuvIQH","ParentId":"a9CF0000000PBTtMAO","CreatedDate":"2015-08-14T20:32:45.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000MKCaN6IQL"},"Id":"017F00000MKCaN6IQL","ParentId":"a9CF0000000PBTyMAO","CreatedDate":"2015-08-14T20:34:02.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000MKCfqsIQD"},"Id":"017F00000MKCfqsIQD","ParentId":"a9CF0000000PBU3MAO","CreatedDate":"2015-08-14T20:34:26.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000MKBPzvIQH"},"Id":"017F00000MKBPzvIQH","ParentId":"a9CF0000000PBTjMAO","CreatedDate":"2015-08-14T19:54:29.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000UpbR6fIQE"},"Id":"017F00000UpbR6fIQE","ParentId":"a9CF00000008P4dMAE","CreatedDate":"2016-07-08T18:58:54.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0172A0000dPqFUHQA3"},"Id":"0172A0000dPqFUHQA3","ParentId":"a9C2A000000KzXkUAK","CreatedDate":"2018-02-01T17:01:35.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0172A0000dQcNrEQAV"},"Id":"0172A0000dQcNrEQAV","ParentId":"a9C2A000000KzYJUA0","CreatedDate":"2018-02-02T17:48:52.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0172A0000dd2OQ9QAM"},"Id":"0172A0000dd2OQ9QAM","ParentId":"a9C2A000000PBhNUAW","CreatedDate":"2018-02-16T00:14:24.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0172A0000dkdFSHQA2"},"Id":"0172A0000dkdFSHQA2","ParentId":"a9C2A000000PBiBUAW","CreatedDate":"2018-02-25T17:30:37.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0172A0000dtveDyQAI"},"Id":"0172A0000dtveDyQAI","ParentId":"a9C2A000000PBikUAG","CreatedDate":"2018-03-07T18:00:08.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JVVNyzIQH"},"Id":"017F00000JVVNyzIQH","ParentId":"a9CF0000000PBAcMAO","CreatedDate":"2015-04-17T17:55:31.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JVVNz0IQH"},"Id":"017F00000JVVNz0IQH","ParentId":"a9CF0000000PBAdMAO","CreatedDate":"2015-04-17T17:55:31.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000JVXZjpIQH"},"Id":"017F00000JVXZjpIQH","ParentId":"a9CF0000000PBAhMAO","CreatedDate":"2015-04-17T20:01:55.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000P7rOOpIQM"},"Id":"017F00000P7rOOpIQM","ParentId":"a9CF0000000KyuEMAS","CreatedDate":"2015-12-10T01:08:25.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000R9qSHsIQM"},"Id":"017F00000R9qSHsIQM","ParentId":"a9CF00000008OWqMAM","CreatedDate":"2016-02-29T21:27:48.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0170Z0000exGsX1QAK"},"Id":"0170Z0000exGsX1QAK","ParentId":"a9C0Z0000008PVOUA2","CreatedDate":"2018-10-15T23:09:28.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000NoEk4xIQC"},"Id":"017F00000NoEk4xIQC","ParentId":"a9CF0000000blLjMAI","CreatedDate":"2015-10-15T01:12:12.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0170Z0000fAxKbjQAF"},"Id":"0170Z0000fAxKbjQAF","ParentId":"a9C0Z0000008PViUAM","CreatedDate":"2018-10-24T17:18:45.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000LrqPhlIQE"},"Id":"017F00000LrqPhlIQE","ParentId":"a9CF0000000PBR4MAO","CreatedDate":"2015-07-27T12:15:12.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/017F00000MfoqasIQA"},"Id":"017F00000MfoqasIQA","ParentId":"a9CF0000000XZARMA4","CreatedDate":"2015-08-29T02:50:22.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0171Y0000sc0832QAA"},"Id":"0171Y0000sc0832QAA","ParentId":"a9C1Y0000004E4EUAU","CreatedDate":"2019-11-19T17:47:37.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0171Y0000sc2DUJQA2"},"Id":"0171Y0000sc2DUJQA2","ParentId":"a9C1Y0000004E4JUAU","CreatedDate":"2019-11-19T20:54:33.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0171Y0000t3TnLnQAK"},"Id":"0171Y0000t3TnLnQAK","ParentId":"a9C1Y0000004E65UAE","CreatedDate":"2019-12-04T21:07:44.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0171Y0000tEZI96QAH"},"Id":"0171Y0000tEZI96QAH","ParentId":"a9C1Y0000004E6jUAE","CreatedDate":"2019-12-10T18:07:38.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0171Y0000tTJipbQAD"},"Id":"0171Y0000tTJipbQAD","ParentId":"a9C1Y0000004E73UAE","CreatedDate":"2019-12-16T15:32:34.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0171Y0000uBnpB2QAJ"},"Id":"0171Y0000uBnpB2QAJ","ParentId":"a9C1Y0000004E7hUAE","CreatedDate":"2020-01-05T10:30:02.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0171Y0000uvNIO2QAO"},"Id":"0171Y0000uvNIO2QAO","ParentId":"a9C1Y0000004E8uUAE","CreatedDate":"2020-01-22T23:35:32.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0171Y0000lI4QbtQAF"},"Id":"0171Y0000lI4QbtQAF","ParentId":"a9C1Y000000GpKxUAK","CreatedDate":"2019-02-08T00:36:34.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0171Y0000lFFgbLQAT"},"Id":"0171Y0000lFFgbLQAT","ParentId":"a9C1Y000000GpKsUAK","CreatedDate":"2019-02-06T00:39:40.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0171Y0000nPPGxxQAH"},"Id":"0171Y0000nPPGxxQAH","ParentId":"a9C1Y000000fxVFUAY","CreatedDate":"2019-04-29T18:30:29.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0171Y0000sxU133QAC"},"Id":"0171Y0000sxU133QAC","ParentId":"a9C1Y0000004E5WUAU","CreatedDate":"2019-12-02T16:34:59.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0171Y0000nqCB9VQAW"},"Id":"0171Y0000nqCB9VQAW","ParentId":"a9C1Y000000fxVtUAI","CreatedDate":"2019-05-15T23:31:02.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0171Y0000oE0KENQA3"},"Id":"0171Y0000oE0KENQA3","ParentId":"a9C1Y000000fxWSUAY","CreatedDate":"2019-05-31T19:52:42.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0171Y0000pBClqEQAT"},"Id":"0171Y0000pBClqEQAT","ParentId":"a9C1Y000000fxgDUAQ","CreatedDate":"2019-07-11T18:18:41.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0171Y0000q9LLKbQAO"},"Id":"0171Y0000q9LLKbQAO","ParentId":"a9C1Y000000fxh6UAA","CreatedDate":"2019-08-19T19:23:57.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0171Y0000qYmFzTQAV"},"Id":"0171Y0000qYmFzTQAV","ParentId":"a9C1Y000000fxj2UAA","CreatedDate":"2019-09-05T19:31:34.000+0000","Field":"created","OldValue":null,"NewValue":null},{"attributes":{"type":"Incident__History","url":"/services/data/v48.0/sobjects/Incident__History/0171Y0000skFQNgQAO"},"Id":"0171Y0000skFQNgQAO","ParentId":"a9C1Y0000004E5RUAU","CreatedDate":"2019-11-23T23:07:46.000+0000","Field":"created","OldValue":null,"NewValue":null}][{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zJVoUAM"},"Id":"a1P1Y000007zJVoUAM","CreatedDate":"2019-11-24T01:25:53.000+0000","IncidentId__c":"a9C1Y0000004E5RUAU","UpdateLog__c":"Okta has taken steps to mitigate the issue, and we are seeing improvements in Dashboard and Authentication requests. At this time user provisioning has been disabled, we are working to restore this functionality and investigating the root cause. Further updates will be provided as we continue to progress towards full resolution","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zJVyUAM"},"Id":"a1P1Y000007zJVyUAM","CreatedDate":"2019-11-24T02:57:49.000+0000","IncidentId__c":"a9C1Y0000004E5RUAU","UpdateLog__c":"Resolved: An issue impacting Okta Advanced Server Access has been resolved. During the event, Administrators may have experienced errors accessing the ASA dashboard. Additionally, users may have experienced authentication errors. Okta applied several changes to mitigate the issue and continues to monitor and investigate. Additional root cause information will be provided within 48 business hours.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zIpTUAU"},"Id":"a1P1Y000007zIpTUAU","CreatedDate":"2019-11-06T19:29:05.000+0000","IncidentId__c":"a9C1Y0000004E2wUAE","UpdateLog__c":"RESOLVED: Okta has successfully deployed and verified a fix across all Cells. The identified issue impacted a subset of the users in all cells using legacy protocol sign-on with Office 365 or service accounts. Additional root cause information will provided within 48 hours.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zHjtUAE"},"Id":"a1P1Y000007zHjtUAE","CreatedDate":"2019-10-01T00:07:15.000+0000","IncidentId__c":"a9C1Y0000004DyLUAU","UpdateLog__c":"We are continuing to work on a workaround. \r\n\r\nPlease note that this issue is specific to Google Chrome version 77 and does not impact Mozilla Firefox, Internet Explorer, or Apple Safari.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zHk3UAE"},"Id":"a1P1Y000007zHk3UAE","CreatedDate":"2019-10-01T01:11:23.000+0000","IncidentId__c":"a9C1Y0000004DyLUAU","UpdateLog__c":"Our team is continuing to work on the workaround to resolve this issue.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zHkDUAU"},"Id":"a1P1Y000007zHkDUAU","CreatedDate":"2019-10-01T01:48:25.000+0000","IncidentId__c":"a9C1Y0000004DyLUAU","UpdateLog__c":"Our team is continuing to work on the workaround to resolve this issue.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zHkIUAU"},"Id":"a1P1Y000007zHkIUAU","CreatedDate":"2019-10-01T01:57:22.000+0000","IncidentId__c":"a9C1Y0000004DyLUAU","UpdateLog__c":"Okta will begin deployment tonight and expect to complete the deployment to all cells by Oct 1 2019 7:00 pm PDT.\r\n\r\nUsers who cannot rely on a browser other than Chrome to use Okta, may workaround the issue by disabling Hardware Acceleration in Chrome Advanced/System Preferences and relaunching browser\r\n\r\nOkta is working with Google to confirm the suspected browser issue resolution\r\n\r\nWe will continue to update as the workaround is deployed to specific Cells.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zHkSUAU"},"Id":"a1P1Y000007zHkSUAU","CreatedDate":"2019-10-01T03:49:19.000+0000","IncidentId__c":"a9C1Y0000004DyLUAU","UpdateLog__c":"Workaround has been deployed to Cell EU1. We will continue to update as the workaround is deployed to other cells.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zHkXUAU"},"Id":"a1P1Y000007zHkXUAU","CreatedDate":"2019-10-01T05:06:40.000+0000","IncidentId__c":"a9C1Y0000004DyLUAU","UpdateLog__c":"Workaround has been deployed to Cell OK1. We will continue to update as the workaround is deployed to other cells.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zHnRUAU"},"Id":"a1P1Y000007zHnRUAU","CreatedDate":"2019-10-01T19:38:20.000+0000","IncidentId__c":"a9C1Y0000004DyLUAU","UpdateLog__c":"Workaround has been deployed to Cell OK8 and Ok9. We will continue to update as the workaround is deployed to the remaining cells.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zHnbUAE"},"Id":"a1P1Y000007zHnbUAE","CreatedDate":"2019-10-01T21:26:19.000+0000","IncidentId__c":"a9C1Y0000004DyLUAU","UpdateLog__c":"Workaround has been deployed to Cell OK5. We will continue to update as the workaround is deployed to the remaining cells.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zHngUAE"},"Id":"a1P1Y000007zHngUAE","CreatedDate":"2019-10-01T22:27:04.000+0000","IncidentId__c":"a9C1Y0000004DyLUAU","UpdateLog__c":"Workaround has been deployed to Cell OK2. We will continue to update as the workaround is deployed to the remaining cells.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zHnvUAE"},"Id":"a1P1Y000007zHnvUAE","CreatedDate":"2019-10-02T00:29:07.000+0000","IncidentId__c":"a9C1Y0000004DyLUAU","UpdateLog__c":"Workaround has been deployed to Cell OK7. We will continue to update as the workaround is deployed to other cells.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zHnqUAE"},"Id":"a1P1Y000007zHnqUAE","CreatedDate":"2019-10-01T23:39:50.000+0000","IncidentId__c":"a9C1Y0000004DyLUAU","UpdateLog__c":"Workaround has been deployed to Cell OK4. We will continue to update as the workaround is deployed to other cells.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zHnlUAE"},"Id":"a1P1Y000007zHnlUAE","CreatedDate":"2019-10-01T23:06:47.000+0000","IncidentId__c":"a9C1Y0000004DyLUAU","UpdateLog__c":"Workaround has been deployed to Cell OK3. We will continue to update as the workaround is deployed to other cells.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zJmaUAE"},"Id":"a1P1Y000007zJmaUAE","CreatedDate":"2019-12-10T19:09:02.000+0000","IncidentId__c":"a9C1Y0000004E6jUAE","UpdateLog__c":"Resolved: Okta has resolved an issue where a subset of customers were unable to authenticate successfully via Agentless Desktop SSO in US Cell 1, US Cell 11, and EMEA Cell 1. Impacted customers may be required to clear their DNS cache to fully restore end user normalcy. Root Cause Analysis will be provided here within 2 business days.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zJmVUAU"},"Id":"a1P1Y000007zJmVUAU","CreatedDate":"2019-12-10T18:40:31.000+0000","IncidentId__c":"a9C1Y0000004E6jUAE","UpdateLog__c":"Okta is currently experiencing an issue with Agentless Desktop SSO for a subset of customers in US Cell 1, US Cell 11, and EMEA Cell 1 resulting in users being redirected to the default login page rather then being authenticated via Agentless Desktop SSO. We are currently taking mitigating actions for the impacted customers","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zJPMUA2"},"Id":"a1P1Y000007zJPMUA2","CreatedDate":"2019-11-20T06:00:37.000+0000","IncidentId__c":"a9C1Y0000004E4JUAU","UpdateLog__c":"Resolved: Okta has addressed an issue on Okta Mobile for Android for the vast majority of Okta end users. When opening the Okta Mobile for Android application, end users may be logged out of their account and encounter the following message: “Detected possible malicious activity when establishing a secure connection to Okta”. Okta Mobile Android users on Android 6.0 or higher must update the Okta Mobile application to the latest version through the Google Play Store. Okta continues to investigate a solution for users running versions of Android below 6.0 as well as those relying on downloading the Okta Mobile APK from their Okta tenant. We apologize for the inconvenience. Additional root cause information will be provided within 48 business hours.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zJOsUAM"},"Id":"a1P1Y000007zJOsUAM","CreatedDate":"2019-11-19T23:10:01.000+0000","IncidentId__c":"a9C1Y0000004E4JUAU","UpdateLog__c":"Investigation is underway for the intermittent issue that Okta reproduced. Customers may see the following error: “Detected possible malicious activity when establishing a secure connection to Okta” when installing or opening the Okta Mobile application on Android. We will provide more information once available.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zJhQUAU"},"Id":"a1P1Y000007zJhQUAU","CreatedDate":"2019-12-02T17:12:06.000+0000","IncidentId__c":"a9C1Y0000004E5WUAU","UpdateLog__c":"Okta is seeing delays in processing of jobs within a subset of our job framework in US Cell 7. We are actively working to mitigate job processing delays and will continue to provide updates as they are available.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000005V3R2UAK"},"Id":"a1P1Y000005V3R2UAK","CreatedDate":"2018-12-13T13:40:43.000+0000","IncidentId__c":"a9C1Y000000GpJuUAK","UpdateLog__c":"An issue impacting CORS authentication in Okta Preview Cells has been identified and customers using OIDC applications and API AM might be impacted. A fix has been identified and we are working to return the service to normal as quickly as possible.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000005V3RCUA0"},"Id":"a1P1Y000005V3RCUA0","CreatedDate":"2018-12-13T14:51:42.000+0000","IncidentId__c":"a9C1Y000000GpJuUAK","UpdateLog__c":"Resolved: The issue impacting CORS authentication for customers using OIDC applications and API AM in Okta Preview Cells has been fixed. Service has been restored to normal.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zJkPUAU"},"Id":"a1P1Y000007zJkPUAU","CreatedDate":"2019-12-08T03:04:04.000+0000","IncidentId__c":"a9C1Y0000004E5WUAU","UpdateLog__c":"Root Cause Analysis: On December 2nd at approximately 7:45AM (PST), Okta detected a slowdown with processing import jobs on US Cell 7 and subsequently determined that there was an unusual and significant increase in asynchronous job workload which resulted in a backup of job processing.\r\n\r\nOkta increased system resources to expedite the processing of pending jobs. This mitigated the issue and normal functionality was fully restored by ‪10:16 AM.\r\n\r\nOkta is working optimize the current implementation of group application assignments to batch and de-duplicate assignments. Okta is also adding monitoring and controls to detect and mitigate similar issues in the future","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zJkUUAU"},"Id":"a1P1Y000007zJkUUAU","CreatedDate":"2019-12-08T04:10:43.000+0000","IncidentId__c":"a9C1Y0000004E65UAE","UpdateLog__c":"Root Cause Analysis: On December 4th 2019 at 11:16am PST, customers on US Cell 3 began reporting issues with authentication and user provisioning after Workday imports. Upon investigation, Okta determined that a subset of customers on US Cell 3 who imported groups to Okta from Workday, Ultipro, or ServiceNow after 10am PST experienced a loss of group membership status for some users after import.\r\n\r\nIn cases where group membership was tied to provisioning for downstream applications, the loss of group memberships also removed user access to those applications. For impacted customers that provisioned Workday users to Active Directory (AD), where group members were part of delegated authentication to AD, affected users were unable to log into Okta.\r\n\r\nAt approximately 1:10pm PST, Okta took action to prevent further customer impact by disabling user import jobs on US Cell 3. Additional investigation determined the observed behavior was a result of an in-progress infrastructure change for existing customers in US Cell 3, which began rolling out at approximately 10am PST. At 1:20pm PST Okta took action to roll back this change and at 1:30 PST import jobs were re-enabled on US Cell 3. Following these updates, Okta published remediation steps and worked directly with the small subset of customers experiencing on-going impact to fully restore user provisioning and group assignments.\r\n\r\nSite functionality was restored at 2:20pm PST. However, additional action was required for a small subset of customers to restore functionality for downstream applications.\r\n\r\nInfrastructure changes were deployed as a phased roll-out within Preview Cells starting March 2019 and enabled for new customers across Production Cells on November 4, 2019. After several months of validation with no issues reported, these changes were approved for deployment on US Cells with existing customers. Despite pre-release testing and a phased roll-out approach to mitigate risk for existing customers, Okta had a testing gap for migrating existing customers running import jobs for group assignment.\r\n\r\nOkta is rigorously reviewing test coverage for import scenarios and working directly with customers to further identify complex use cases with downstream dependencies to enhance our test suites. As part of our on-going focus on reliability, Okta is also working to implement more granular monitoring and automated mitigation mechanisms to protect against similar issues during future infrastructure updates.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zIsSUAU"},"Id":"a1P1Y000007zIsSUAU","CreatedDate":"2019-11-10T17:26:13.000+0000","IncidentId__c":"a9C1Y0000004E2wUAE","UpdateLog__c":"Root Cause Analysis:\r\n\r\nOn Nov 6th at approximately 9:30 AM PDT, Okta received reports from certain customers across multiple cells that some users were not able to log into the Okta Service via Office 365 clients using legacy protocols such as the Outlook Desktop App, and some service accounts used for automation.\r\n\r\nOnly end users using Outlook clients with an Exchange ActiveSync profile and without an enrolled second factor that are members of tenants with a configured MFA Enrollment Policy were impacted.\r\n\r\nThe pursuant investigation determined that a bug fix, applied in the 2019.10.2 weekly update, to address a gap in multi-factor enrollment during authentication, was the root cause. The fix was incorrectly scoped to include legacy Office 365 protocols.\r\n\r\nAt approximately 10:44 AM PDT, Okta reverted the change in US Cell 4 and confirmed that issue was resolved. The change was then subsequently reverted in all production. At approximately 11:35 AM, service was fully restored for all customers in all cells.\r\n\r\nAdditional testing and alerting to prevent and detect such issues will be implemented to prevent future occurrences.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zJOnUAM"},"Id":"a1P1Y000007zJOnUAM","CreatedDate":"2019-11-19T23:04:32.000+0000","IncidentId__c":"a9C1Y0000004E4EUAU","UpdateLog__c":"Okta continues to work with Zoom to resolve this issue. We are actively working to deploy a fix to allow provisioning to complete without error. Zoom has in-tandem confirmed they will be working implement a fix for the underlying issue later today.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zJiEUAU"},"Id":"a1P1Y000007zJiEUAU","CreatedDate":"2019-12-03T21:50:03.000+0000","IncidentId__c":"a9C1Y0000004E5RUAU","UpdateLog__c":"Root Cause Analysis: On Saturday, November 23rd at approximately 1:05 PM PDT, Okta detected database resource contention within our Advanced Server Access production environment, resulting in timeout errors for users who were accessing the Advanced Server Access dashboard or were attempting to authenticate to servers.\r\n\r\nAfter investigating the issue, at approximately 3:30 PM PDT, Okta took actions to mitigate the resource contention issue by blocking customer API requests for adding groups to projects and synchronizing data to servers. Most Advanced Server Access functionality was restored through this action. Upon further investigation, Okta identified an additional inefficiency in our synchronous logic, and made a change to asynchronous background job processing.\r\n\r\nFollowing our investigation and mitigation actions, Okta re-enabled the API for synchronizing data to servers at approximately 6:47 PM PDT. Okta also re-enabled the API for adding groups to projects, fully restoring all services at this time.\r\n\r\nOkta's has implemented changes to mitigate the impact of similar issues in the future, and is continuing its full Root Cause Analysis of the incident,. We expect to be able to provide further details and future preventative actions by December 4, 2019. Additional RCA details will be posted here once this is investigation is complete.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000005V3RHUA0"},"Id":"a1P1Y000005V3RHUA0","CreatedDate":"2018-12-17T07:38:38.000+0000","IncidentId__c":"a9C1Y000000GpJuUAK","UpdateLog__c":"Root Cause Analysis: On December 12th, an issue was introduced in Okta Preview cells which impacted the use of the Auth SDKs. Specifically, customers using version 2.0 or higher of Okta’s JavaScript SDK AuthJS (@okta/okta-auth-js) to leverage OAuth 2.0 may have experienced authentication failures starting at approximately 2:21pm PST. Additionally, customers using Okta’s Angular, React and Vue integration libraries and OAuth 2.0 in Preview cells were also impacted by this issue. The degree of impact varied depending on each customers’ implementation and client-side caching. \r\n<br><br>\r\nThe root cause was traced to an unexpected side effect from a fix applied to our CORS/TrustedOrigins framework. The issue was fully resolved on Dec 13th by 6:26am PST in US Preview Cell 1 and by 6:38am PST in EU Preview Cell 2. To prevent a similar re-occurrence, the test automation framework gap has been updated and existing test suites are being enhanced to identify such errors in the future..","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zJhaUAE"},"Id":"a1P1Y000007zJhaUAE","CreatedDate":"2019-12-02T18:23:57.000+0000","IncidentId__c":"a9C1Y0000004E5WUAU","UpdateLog__c":"An issue impacting job processing for a subset of customers within US Cell 7 has been resolved. The remaining queued jobs are processing at full capacity and we will continue to monitor. Root Cause Analysis is underway and we will post full RCA details here within 48 hours.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zJMrUAM"},"Id":"a1P1Y000007zJMrUAM","CreatedDate":"2019-11-19T18:32:32.000+0000","IncidentId__c":"a9C1Y0000004E4EUAU","UpdateLog__c":"Okta customers in all cells that leverage Zoom’s integration for provisioning are experiencing errors while updating users in Zoom. We are still working with Zoom to investigate the errors and will provide another update in 2 hours.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zJOYUA2"},"Id":"a1P1Y000007zJOYUA2","CreatedDate":"2019-11-19T20:35:05.000+0000","IncidentId__c":"a9C1Y0000004E4EUAU","UpdateLog__c":"Zoom confirmed that a recent release to add an email update feature resulted in a change to the provisioning API. This change has impacted Okta customers in all cells that leverage Okta's Zoom integration for provisioning - customers are seeing errors while updating users in Zoom. We are continuing to work with Zoom to identify a solution and timeline. We will provide the next update in 2 hours.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zJOdUAM"},"Id":"a1P1Y000007zJOdUAM","CreatedDate":"2019-11-19T22:00:06.000+0000","IncidentId__c":"a9C1Y0000004E4JUAU","UpdateLog__c":"Okta is able to reproduce, and continues to investigate, an intermittent issue presenting users with the error “Detected possible malicious activity when establishing a secure connection to Okta” when installing or opening the Okta Mobile application on Android. We will provide more information once available.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zJOiUAM"},"Id":"a1P1Y000007zJOiUAM","CreatedDate":"2019-11-19T22:34:57.000+0000","IncidentId__c":"a9C1Y0000004E4JUAU","UpdateLog__c":"Okta was able to reproduce an intermittent issue presenting users with the error “Detected possible malicious activity when establishing a secure connection to Okta” when installing or opening the Okta Mobile application on Android. We are continuing to investigate and will provide more information once available.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zJOxUAM"},"Id":"a1P1Y000007zJOxUAM","CreatedDate":"2019-11-19T23:55:49.000+0000","IncidentId__c":"a9C1Y0000004E4JUAU","UpdateLog__c":"Okta has determined the root cause of the issue on Android when installing or launching Okta Mobile or Okta Verify. Okta Engineering is actively working on a fix and will release new versions of Okta Mobile and Okta Verify as soon as they are ready. While the message displayed in the app may sound alarming, the issue is benign, and due to our over abundance of caution with TLS algorithm restrictions.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zJP7UAM"},"Id":"a1P1Y000007zJP7UAM","CreatedDate":"2019-11-20T03:05:37.000+0000","IncidentId__c":"a9C1Y0000004E4JUAU","UpdateLog__c":"Okta has determined the root cause of the issue on Android when installing or launching Okta Mobile or Okta Verify. Okta Engineering is actively working on a fix and expects to begin roll-out of the new versions of Okta Mobile and Okta Verify within the next hour. While the message displayed in the app may sound alarming, the issue is benign, and due to our over abundance of caution with TLS algorithm restrictions.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zJP2UAM"},"Id":"a1P1Y000007zJP2UAM","CreatedDate":"2019-11-20T00:41:22.000+0000","IncidentId__c":"a9C1Y0000004E4JUAU","UpdateLog__c":"Okta has determined the root cause of the issue on Android when installing or launching Okta Mobile or Okta Verify. Okta Engineering is actively working on a fix and will release new versions of Okta Mobile and Okta Verify as soon as they are ready. While the message displayed in the app may sound alarming, the issue is benign, and due to our over abundance of caution with TLS algorithm restrictions. The next update will be posted in approximately 2 hours.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zJPCUA2"},"Id":"a1P1Y000007zJPCUA2","CreatedDate":"2019-11-20T04:30:22.000+0000","IncidentId__c":"a9C1Y0000004E4JUAU","UpdateLog__c":"Okta has determined the root cause of the issue on Android when installing or launching Okta Mobile. Okta Engineering is actively working on a fix and expects to release the new version of Okta Mobile within the hour. Okta has determined there is no meaningful impact to Okta Verify at this point. Additionally, while the message displayed in the Okta Mobile app may sound alarming, the issue is benign, and due to our over abundance of caution with TLS algorithm restrictions.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zJPHUA2"},"Id":"a1P1Y000007zJPHUA2","CreatedDate":"2019-11-20T05:02:16.000+0000","IncidentId__c":"a9C1Y0000004E4EUAU","UpdateLog__c":"Resolved: An issue impacting user provisioning updates to the Zoom application within Okta has been resolved. During the event, user updates to the Zoom application may have encountered errors if the provisioned user already existed within Zoom. Okta worked with Zoom to identify the root cause of the issue and applied a hot-fix to all production cells to mitigate the provisioning errors while Zoom worked to deploy a permanent fix. Additional root cause information will be provided within 48 business hours.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zJV0UAM"},"Id":"a1P1Y000007zJV0UAM","CreatedDate":"2019-11-22T17:31:58.000+0000","IncidentId__c":"a9C1Y0000004E4EUAU","UpdateLog__c":"Root Cause Analysis: On Nov 19th at approximately 8:53AM PDT, customers reported Zoom provisioning errors when updating user’s profiles.\r\n\r\nUpon investigation, Okta determined that the issue was caused by a change in the behavior of an existing Zoom Provisioning API which was introduced on November 17. This change was implemented by Zoom to enable support for changing user's email addresses through the Profile Update request. The change inadvertently caused profile updates to fail, whenever the email has not been changed.\r\n\r\nOn Nov 19th at approximately 8:17 PM PDT Okta released a work around to address the issue. This update was deployed to all production cells and service was fully restored.\r\n\r\nOkta is working closely with Zoom for a fix, as well as, understanding and mitigating the re-occurrence for this type of failure","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zJqIUAU"},"Id":"a1P1Y000007zJqIUAU","CreatedDate":"2019-12-16T08:19:41.000+0000","IncidentId__c":"a9C1Y0000004E6jUAE","UpdateLog__c":"Root Cause Analysis: On December 10th, 2019 at approximately 9:30 AM PST, Okta began investigating reports by customers of problems authenticating via Agent-less Desktop SSO in US Cell 1, US Cell 11, and EU Cell 1. \r\n\r\nOkta determined that during the implementation of new load-balancers on December 4th 2019 at 3:15pm PST, DNS entries for Kerberos endpoints were not updated correctly for the impacted cells. As a result, Agent-less Desktop SSO traffic was still directed to the previous load balancers which were no longer working as intended, resulting in requests that timed out and user browser sessions being redirected to Okta login screens. While affected users were able to login, they had to type in their credentials instead of being logged in automatically.\r\n\r\nAt approximately 10:48 AM, Okta updated the affected entries to reference the new load-balancers and service was restored. Some clients may have had to refresh DNS caches to restore service more quickly.\r\n\r\nOkta is working to implement additional monitoring and automation of changes to DNS records to prevent similar issues during future updates.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zJtgUAE"},"Id":"a1P1Y000007zJtgUAE","CreatedDate":"2019-12-19T02:29:57.000+0000","IncidentId__c":"a9C1Y0000004E73UAE","UpdateLog__c":"Root Cause Analysis: On December 16, 2019 at approximately 6:22 AM PST, Okta detected a degradation in application performance in US Cell 4. As a result, customers may have experienced:\r\n- Longer than normal response times\r\n- A small percentage (0.25%) experienced sporadic read-only errors\r\n- Admin users may have intermittently seen the read-only banner\r\n\r\nUpon investigation, Okta determined that the root cause was unusual database activity required to process a significant number of group rules and assignments. Although our throttling and rate limiting mechanisms engaged, a portion of requests saw additional latency due to the increased utilization of our database tier while the load was being redistributed.\r\nAt approximately 6:48 AM PST, Okta began to redistribute requests across the cell’s database cluster to reduce load and contention on the impacted database. By 8:09 AM PST, normal service performance was fully restored.\r\n\r\nOkta is implementing additional monitoring, optimization, and mitigation strategies to prevent similar issues in the future. Okta is also investigating asynchronous processing for group rules and performance improvements for group deletions to prevent system slowdown.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zK0rUAE"},"Id":"a1P1Y000007zK0rUAE","CreatedDate":"2020-01-08T14:18:37.000+0000","IncidentId__c":"a9C1Y0000004E7hUAE","UpdateLog__c":"Root Cause Analysis: On January 5th at approximately 1:19am PST Okta received an alert that an automated fail-over had occurred within our caching infrastructure in US Cell 4.\r\n\r\nInitial investigation determined that a cache infrastructure server in one availability zone had experienced a brief network outage. As a result, a secondary cache node in a different availability zone was automatically promoted to primary. Manual analysis of slot and node health indicated that all nodes in the cluster were healthy but that cluster resiliency was degraded as a result of the fail-over.\r\n\r\nIn order to regain cluster resiliency, a fail-back command to revert to the original primary/secondary node relationship was manually issued at 1:39am PST. This fail-back attempt completed. However, immediately thereafter all requests to the specific hash range served by this node either timed out if the primary was contacted directly or errored out if the request hit another node of the cluster.\r\n\r\nThe combination of timeouts, retries, and new requests resulted in customers in US Cell 4 experiencing increased latency and intermittent 429 (rate limit) or 502 (server resource unavailable) error responses.\r\n\r\nFurther investigation determined the original network outage and corresponding secondary node promotion was due to a local hardware issue on the cache cluster primary node from the underlying cloud network service provider. Cluster health checks performed prior to fail-back did not capture this underlying hardware instability, which was on-going. Due to this oversight, the fail-back command resulted in a primary node assignment within an unstable environment. After the fail-back to the original but unstable server was complete, the original primary node became unresponsive and was unable to fail-over, either automatically or manually.\r\n\r\nSuch a scenario was not covered by our Operational Run-books, and a cache cluster specialist was engaged. At approximately 2:17 AM PST, Okta succeeded in initiating a forced fail-over within US Cell 4 to regain cluster stability and restore service. Service was fully restored at 2:38 AM PST.\r\n\r\nAs a result of this outage, Okta is upgrading run-books to cover non-responsive primaries and is evaluating training, specialization, and follow the sun coverage to resolve specific cache cluster recovery procedures when automatic fail-over is not possible.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zK4PUAU"},"Id":"a1P1Y000007zK4PUAU","CreatedDate":"2020-01-23T00:06:16.000+0000","IncidentId__c":"a9C1Y0000004E8uUAE","UpdateLog__c":"Okta is investigating to restore the incorrect AD Agent state information being reported. Customers with IWA automatic fail-over failing can change to manual fail-over mode to return IWA authentication to service","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zK4UUAU"},"Id":"a1P1Y000007zK4UUAU","CreatedDate":"2020-01-23T00:43:57.000+0000","IncidentId__c":"a9C1Y0000004E8uUAE","UpdateLog__c":"Okta is actively working to restore the incorrect AD Agent state information being reported. While being reported as NOT_CONNECTED, monitoring shows that AD Agent authentication continues to be successful.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P0Z000007eNdsUAE"},"Id":"a1P0Z000007eNdsUAE","CreatedDate":"2018-05-17T19:46:07.000+0000","IncidentId__c":"a9C0Z000000TNgDUAW","UpdateLog__c":"An issue impacting authentication was resolved at 12:19pm PDT. During the event, End-Users and Administrators may have experienced HTTP 5xx errors while attempting to authenticate to their Okta tenant. Additional root cause information will be provided within 48 hours.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P0Z000007eOBkUAM"},"Id":"a1P0Z000007eOBkUAM","CreatedDate":"2018-05-20T09:27:36.000+0000","IncidentId__c":"a9C0Z000000TNgDUAW","UpdateLog__c":"On May 17th, beginning at 11:28am PDT, Okta experienced a service disruption in US Cell 6. End-User and Administrators attempting to authenticate to their Okta tenant may have experienced HTTP 5xx errors. Through the duration of the incident, approximately 30% of US Cell 6 users attempting primary or multi-factor authentication encountered errors and would have needed to retry. Other requests, and End-Users and Administrators who already had an active session did not experience any impact.\r\n<br><br>\r\nThe issue surfaced when the cell's session token management cluster became overwhelmed by a sudden increase in the number of active sessions. Automated monitoring failed to effectively alert before the cluster was nearing its maximum capacity.\r\n<br><br>\r\nThe issue was resolved by 12:19pm PDT after operational actions were taken to manage the session count while capacity was added to the cluster. Immediately following the incident, Okta corrected the monitoring gap to prevent a future recurrence. Architectural design work is being initiated to better manage session counts and their expiry.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P2A00000FgbdXUAR"},"Id":"a1P2A00000FgbdXUAR","CreatedDate":"2018-03-20T11:30:33.000+0000","IncidentId__c":"a9C2A0000004CSTUA2","UpdateLog__c":"Root Cause Analysis: \r\n\r\nOn March 16th, beginning at 8:23 am PDT, Okta experienced an issue impacting US Cell 6 Administrators' ability to view recently processed user updates in the admin dashboard. The dashboard queries data from secondary read-only databases which were impacted by a replication error. \r\n\r\nThe root cause was traced to an internal database engine error on the primary database which required a consistency check on the secondaries. Addressing the replication failure and resultant latency was delayed because its alert was superseded by alerts for the database engine error. Shortly after the replication latency was escalated by Support, the issue was cleared by firstly re-directing the impacted traffic to the primary databases, and secondly by repairing the replication lag. By 10:22am PDT, Administrator dashboards displayed all current data. Okta is implementing recurrent alerting for replication lags to ensure faster response times going forward.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P2A00000Fgbi3UAB"},"Id":"a1P2A00000Fgbi3UAB","CreatedDate":"2018-03-20T22:34:12.000+0000","IncidentId__c":"a9C2A0000004CSnUAM","UpdateLog__c":"Okta continues to investigate an issue with Provisioning Setup and IDP-Initiated Single Sign-On functionality for the following Office 365 Apps: Yammer, CRM and Teams. SP-Initiated Single-Sign-On is not affected. Please contact support if you are having difficulty setting up provisioning and we will assist with implementation of an applicable workaround as we investigate further.<br><br>","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P2A00000Fgbi8UAB"},"Id":"a1P2A00000Fgbi8UAB","CreatedDate":"2018-03-20T23:25:38.000+0000","IncidentId__c":"a9C2A0000004CSnUAM","UpdateLog__c":"Okta continues to investigate an issue with Provisioning Setup and IDP-Initiated Single Sign-On functionality for the following Office 365 Apps: Yammer, CRM and Teams. SP-Initiated Single-Sign-On is not affected. Please contact support if you are having difficulty setting up provisioning and we will assist with implementation of an applicable workaround as we investigate further. Another update is planned for 6:25pm PT or sooner as additional information becomes available.<br><br>","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P2A00000FgbiDUAR"},"Id":"a1P2A00000FgbiDUAR","CreatedDate":"2018-03-21T01:27:26.000+0000","IncidentId__c":"a9C2A0000004CSnUAM","UpdateLog__c":"Okta continues to investigate an issue with Provisioning Setup and IDP-Initiated Single Sign-On functionality for the following Office 365 Apps: Yammer, CRM and Teams. SP-Initiated Single-Sign-On is not affected. Please contact support if you are having difficulty setting up provisioning and we will assist with implementation of an applicable workaround as we investigate further. Another update is planned for 7:00pm PT or sooner as additional information becomes available.<br><br>","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P2A00000FgbiNUAR"},"Id":"a1P2A00000FgbiNUAR","CreatedDate":"2018-03-21T02:25:27.000+0000","IncidentId__c":"a9C2A0000004CSnUAM","UpdateLog__c":"Okta continues to investigate an issue with Provisioning Setup and IDP-Initiated Single Sign-On functionality for the following Office 365 Apps: Yammer, CRM and Teams. SP-Initiated Single-Sign-On is not affected. Please contact support if you are having difficulty setting up provisioning and we will assist with implementation of an applicable workaround as we investigate further. Another update is planned for 9:00pm PT or sooner as additional information becomes available.<br><br>","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P2A00000FgbiSUAR"},"Id":"a1P2A00000FgbiSUAR","CreatedDate":"2018-03-21T03:19:22.000+0000","IncidentId__c":"a9C2A0000004CSnUAM","UpdateLog__c":"Okta is currently working to mitigate the issue with IDP-Initiated Single Sign-On functionality for the following Office 365 Apps: Yammer, CRM and Teams. We are exploring a patch with a workaround while we work on a permanent fix. We will continue to provide updates as plans solidify to patch. In addition, please continue to contact support if you are having difficulty setting up O365 provisioning and we will assist with implementation of an applicable workaround for that problem. Another update is planned for 10:30pm PT or sooner as additional information becomes available.<br><br>","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P2A00000FgbiXUAR"},"Id":"a1P2A00000FgbiXUAR","CreatedDate":"2018-03-21T05:36:54.000+0000","IncidentId__c":"a9C2A0000004CSnUAM","UpdateLog__c":"Okta is currently working to mitigate the issue with IDP-Initiated Single Sign-On functionality for the following Office 365 Apps: Yammer, CRM and Teams. We are actively working on a patch deployment with a workaround while we work on a permanent fix. We will continue to provide updates as plans solidify to patch. In addition, please continue to contact support if you are having difficulty setting up O365 provisioning and we will assist with implementation of an applicable workaround for that problem. Another update is planned for 12:30 am PT on 03/21/2018 or sooner as additional information becomes available.\r\n<br><br>","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P2A00000FgbicUAB"},"Id":"a1P2A00000FgbicUAB","CreatedDate":"2018-03-21T07:33:17.000+0000","IncidentId__c":"a9C2A0000004CSnUAM","UpdateLog__c":"Okta is currently working to mitigate the issue with IDP-Initiated Single Sign-On functionality for the following Office 365 Apps: Yammer, CRM and Teams. We are actively working on deploying a patch to all cells with a workaround while we work on a permanent fix. In addition, please continue to contact support if you are having difficulty setting up O365 provisioning and we will assist with implementation of an applicable workaround for that problem. Another update will be provided as soon all cells have been successfully patched.\r\n<br><br>","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P2A00000FgbimUAB"},"Id":"a1P2A00000FgbimUAB","CreatedDate":"2018-03-21T10:43:22.000+0000","IncidentId__c":"a9C2A0000004CSnUAM","UpdateLog__c":"Okta has completed the deployment of a patch to all affected cells with a workaround to address the issue with IDP-Initiated Single Sign-On functionality for the following Office 365 Apps: Yammer, CRM and Teams. SP initiated authentication was unaffected by this issue. We are working on a permanent fix and will provide root cause analysis once it’s available.<br><br>","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P2A00000Fgbt6UAB"},"Id":"a1P2A00000Fgbt6UAB","CreatedDate":"2018-03-22T21:42:13.000+0000","IncidentId__c":"a9C2A0000004CSnUAM","UpdateLog__c":"Root Cause Analysis:\r\n\r\nOn 3/20/2018 at 9:50am PDT, Okta identified an issue with IDP-Initiated Single Sign-On functionality for the following Office 365 Apps: Yammer, Teams and some CRM instances. SP initiated authentication was unaffected by this issue. In addition, some customers were unable to complete the Advanced API Access setup for newly created Office 365 application instances. The service disruption was caused by a deactivation of an Okta tenant on Azure Active Directory. Okta deployed workaround to all affected cells from 11:32PM to 4:11am PDT on 3/20. The required AAD tenant was recovered on 3/21 which restored full functionality. In addition, we are reevaluating our internal processes with a focus on third party dependencies required for custom sign on protocols such as the ones used by these applications.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P2A00000Fj03vUAB"},"Id":"a1P2A00000Fj03vUAB","CreatedDate":"2017-05-15T13:52:05.000+0000","IncidentId__c":"a9C2A000000blruUAA","UpdateLog__c":"Okta is continuing to investigate the cause of the slow interactive response times in US Cell 2.  Admins may also experience delays in Job processing while we continue our investigation.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P2A00000Fj045UAB"},"Id":"a1P2A00000Fj045UAB","CreatedDate":"2017-05-15T14:19:12.000+0000","IncidentId__c":"a9C2A000000blruUAA","UpdateLog__c":"Interactive user experience has returned to normal in US Cell 2.  Admins may continue to experience delays in job processing while we investigate the root cause.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P2A00000Fj04ZUAR"},"Id":"a1P2A00000Fj04ZUAR","CreatedDate":"2017-05-15T15:03:33.000+0000","IncidentId__c":"a9C2A000000blruUAA","UpdateLog__c":"RESOLVED: 8:02am PDT: An issue impacting system responsiveness, including interactive performance and job processing in US Cell 2 was addressed.  Additional root cause information will be provided here shortly.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P2A00000Fjr2PUAR"},"Id":"a1P2A00000Fjr2PUAR","CreatedDate":"2018-03-30T15:15:50.000+0000","IncidentId__c":"a9C2A000000bmK1UAI","UpdateLog__c":"We are currently working with Microsoft who has acknowledged the issue via incident MO133118. As a workaround, customers can utilize portal.microsoft.com to access the Microsoft Office 365 Portal. More details of Microsoft incident MO133118 below:\r\n<br><br>\r\nhttps://portal.office.com/adminportal/home#/servicehealth. \r\n<br><br>\r\nMO133118 - Unable to access Office 365 portal\r\nStatus:Service degradation\r\nUser impact:Users may be unable to access the Office 365 portal.\r\nLatest message:Title: Unable to access Office 365 portal\r\n\r\nUser Impact: Users may be unable to access the Office 365 portal.\r\n\r\nMore info: Users attempting to access the Office 365 portal via https://portal.microsoftonline.com my receive the following error message: \"This site can't be reached\".\r\n\r\nSome users may be able to access the Office 365 portal via https://portal.office.com.\r\n\r\nCurrent status: We're analysing diagnostic data to isolate the root cause and identify steps to remediate impact.\r\n\r\nScope of impact: This issue could potentially affect any of your users if they are routed through the affected infrastructure.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P2A00000Fjr2ZUAR"},"Id":"a1P2A00000Fjr2ZUAR","CreatedDate":"2018-03-30T15:49:50.000+0000","IncidentId__c":"a9C2A000000bmK1UAI","UpdateLog__c":"Okta is monitoring progress with Microsoft on this issue. Recent mitigating steps have been taken to address the issue and we are actively monitoring this incident until fully resolved.\r\n<br><br>","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P0Z0000051aauUAA"},"Id":"a1P0Z0000051aauUAA","CreatedDate":"2018-05-31T08:06:30.000+0000","IncidentId__c":"a9C0Z0000004E5UUAU","UpdateLog__c":"An issue impacting all functionality in US Cell 6 is being investigated. It appears that the underlying infrastructure issue has been resolved, and we are monitoring closely to ensure service has been restored.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P0Z000005XZAoUAO"},"Id":"a1P0Z000005XZAoUAO","CreatedDate":"2018-09-04T14:37:01.000+0000","IncidentId__c":"a9C0Z0000008PTrUAM","UpdateLog__c":"An issue causing intermittent authentication issues to Microsoft Office 365 for a subset of end-users and administrators in all Cells is being investigated. Microsoft has indicated they are experiencing a data center issue resulting in a degraded Office 365 user experience and are actively working to resolve the issue. Further information is available from Microsoft via [https://status.office365.com](https://status.office365.com).","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P0Z000005XZBwUAO"},"Id":"a1P0Z000005XZBwUAO","CreatedDate":"2018-09-04T22:43:47.000+0000","IncidentId__c":"a9C0Z0000008PTwUAM","UpdateLog__c":"Okta is working with Google to remove the \"deceptive site\" flagging for *.oktapreview.com from the Google Safe Browsing list. We are working to resolve the issue as quickly as possible.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P0Z000005XZCGUA4"},"Id":"a1P0Z000005XZCGUA4","CreatedDate":"2018-09-05T03:16:40.000+0000","IncidentId__c":"a9C0Z0000008PTwUAM","UpdateLog__c":"Resolved: An issue impacting access to *.oktapreview.com due to a \"Deceptive site\" flagging from the Google Safe Browsing list on Google Chrome and Firefox has been resolved. During this event, End-Users and Admins may have encountered a \"Deceptive site ahead\" warning while attempting to login to their Okta Preview tenant. Additional root cause information will provided within 48 hours.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P0Z000005XZCLUA4"},"Id":"a1P0Z000005XZCLUA4","CreatedDate":"2018-09-05T03:53:22.000+0000","IncidentId__c":"a9C0Z0000008PTrUAM","UpdateLog__c":"Resolved: An issue impacting authentication and system responsiveness for a subset of end-users and administrators accessing Microsoft Office 365 has subsided. Further investigation of the issue found the errors were correlated with a service degradation reported by Microsoft. Further information is available from Microsoft via https://status.office365.com.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P0Z000005XZDEUA4"},"Id":"a1P0Z000005XZDEUA4","CreatedDate":"2018-09-05T18:42:35.000+0000","IncidentId__c":"a9C0Z0000008PU1UAM","UpdateLog__c":"An issue causing intermittent authentication to Microsoft Office 365 for a subset of end-users and administrators in all Cells is being investigated. Users attempting to authenticate with Office 365 are receiving a \"Throttled\" message. We are actively working with Microsoft who has reported this is related to mitigation activities from yesterday's service disruption (MO147606):\r\n\r\n\"As part of our follow-up remediation actions stemming from the issue reported under service incident MO147606, an update was introduced to the components that manage authentication. We’ve determined that this update has resulted in users receiving a message indicating they are being throttled when attempting to access Outlook and Skype. We’re reverting the update to remediate the problem”","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P0Z000005XZDOUA4"},"Id":"a1P0Z000005XZDOUA4","CreatedDate":"2018-09-05T19:16:27.000+0000","IncidentId__c":"a9C0Z0000008PU1UAM","UpdateLog__c":"We are actively working with Microsoft who has reported this is related to mitigation activities from yesterday's service disruption (MO147606):\r\n\r\n\r\n\"We identified and reverted an update that was causing multiple services to be throttled. We're conducting tests to ensure that impact has been resolved. Further updates can be found in the admin portal under EX147785 and MO147789.\"\r\n\r\nOkta will post additional information as we learn it from Microsoft.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P0Z000005XZDTUA4"},"Id":"a1P0Z000005XZDTUA4","CreatedDate":"2018-09-05T19:31:37.000+0000","IncidentId__c":"a9C0Z0000008PU1UAM","UpdateLog__c":"Resolved: An issue impacting authentication and system responsiveness for a subset of end-users and administrators accessing Microsoft Office 365 has subsided. Further investigation of the issue found the errors were correlated with a service degradation reported by Microsoft. Microsoft reported this is related to mitigation activities form yesterday's service disruption (MO147606):\r\n\r\n\"We identified and reverted an update that was causing multiple services to be throttled. We're conducting tests to ensure that impact has been resolved. Further updates can be found in the admin portal under EX147785 and MO147789.\"","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P0Z000005XZU5UAO"},"Id":"a1P0Z000005XZU5UAO","CreatedDate":"2018-09-18T01:23:42.000+0000","IncidentId__c":"a9C0Z0000008PUVUA2","UpdateLog__c":"End users are currently experiencing errors when authenticating to O365. We are actively investigating and will update this message with more information as soon as we have it.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P0Z000005XZUAUA4"},"Id":"a1P0Z000005XZUAUA4","CreatedDate":"2018-09-18T01:55:55.000+0000","IncidentId__c":"a9C0Z0000008PUVUA2","UpdateLog__c":"End users are currently experiencing intermittent errors when authenticating to O365. Okta is executing a procedure to resolve the errors with most clients now encountering no errors. We will update this message with more information as soon as we have it.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P0Z000005XaKaUAK"},"Id":"a1P0Z000005XaKaUAK","CreatedDate":"2018-10-15T23:35:51.000+0000","IncidentId__c":"a9C0Z0000008PVOUA2","UpdateLog__c":"An issue impacting authentication and system responsiveness for a subset of end-users and administrators in US Cell 6 is actively being mitigated. System responsiveness and major functionality have returned to normal. We are continuing to investigate the root cause and will continue to provide updates until all issues are fully resolved.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P0Z000005XaKfUAK"},"Id":"a1P0Z000005XaKfUAK","CreatedDate":"2018-10-16T00:08:32.000+0000","IncidentId__c":"a9C0Z0000008PVOUA2","UpdateLog__c":"An issue impacting authentication and system responsiveness for a subset of end-users and administrators in US Cell 6 has been resolved. System responsiveness and major functionality have been restored l. We are currently investigating the root cause and will post the full Root Cause Analysis within the next 48 hours.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P0Z000005XaP6UAK"},"Id":"a1P0Z000005XaP6UAK","CreatedDate":"2018-10-18T04:22:28.000+0000","IncidentId__c":"a9C0Z0000008PVOUA2","UpdateLog__c":"Resolved: On October 15th, beginning at 3:58 PM PDT, Okta experienced a service disruption in US Cell 6. The service disruption occurred during two distinct windows. Between 3:58 PM PDT and 4:05 PM PDT, approximately 15% of incoming end user requests to US Cell 6 experienced latency or errors. Between 4:28 PM PDT and 4:33 PM PDT, approximately 9% of end-user requests to US Cell 6 experienced latency or errors. The service disruption was resolved and the service was returned to normal at 4:33pm PDT.\r\n\r\nThe root cause of the service disruptions was traced to a significant and abnormal increase in requests that activated Okta's rate limiting framework but eventually resulted in higher than normal CPU utilization within our infrastructure. Starting at 3:57, Okta's monitoring services detected the increase in abnormal traffic, and automated service protections were employed to mitigate the impact, however, additional actions were needed to reroute and isolate the offending traffic.\r\n\r\nFollowing the service disruption, Okta reviewed the incident response and has identified and plans to implement additional automated and manual incident response process improvements and infrastructure changes to further minimize impact or prevent similar incidents in the future.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P0Z000005XaYwUAK"},"Id":"a1P0Z000005XaYwUAK","CreatedDate":"2018-10-25T03:43:48.000+0000","IncidentId__c":"a9C0Z0000008PViUAM","UpdateLog__c":"Resolved: An issue impacting compatibility with Office 365 user provisioning has been resolved. Root cause analysis will be posted here within 48 hours.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P0Z000005XasDUAS"},"Id":"a1P0Z000005XasDUAS","CreatedDate":"2018-11-12T00:10:58.000+0000","IncidentId__c":"a9C0Z0000008PW7UAM","UpdateLog__c":"We continue to observe delays or failures in the delivery of outbound SMS to the AT&T network in the US. Most end users should be able to succeed with a retry. SMS sent to other networks are unaffected. We are actively working with our providers to resolve the issue and will provide an update when we have further information. Okta recommends affected AT&T mobile customers use an alternate second factor as a work around to the issue.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P0Z000005Xb1FUAS"},"Id":"a1P0Z000005Xb1FUAS","CreatedDate":"2018-11-14T19:13:55.000+0000","IncidentId__c":"a9C0Z0000008PW7UAM","UpdateLog__c":"Resolved: Okta received reports of delays and failures in the delivery of SMS for multi-factor authentication. Upon further investigation, it was determined that the issue was caused by an on-going AT&T cellular network incident. Okta worked with its service providers to raise the issue and worked with customers to workaround the the issue until service was fully restored by AT&T. Users who were not on the AT&T cellular network or that leveraged other authentication factors were not affected throughout the incident.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P0Z000007eJNEUA2"},"Id":"a1P0Z000007eJNEUA2","CreatedDate":"2018-04-23T16:41:04.000+0000","IncidentId__c":"a9C0Z000000TNelUAG","UpdateLog__c":"Okta is currently investigating an issue with the /api/v1/events endpoint in all cells. To access system logs, please utilize the [System Log API Endpoint](https://developer.okta.com/docs/api/resources/system_log).","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P0Z000007eJO7UAM"},"Id":"a1P0Z000007eJO7UAM","CreatedDate":"2018-04-23T17:13:24.000+0000","IncidentId__c":"a9C0Z000000TNelUAG","UpdateLog__c":"Okta is currently investigating an issue with the /api/v1/events endpoint in all cells. To access system logs, please utilize the [System Log API Endpoint](https://developer.okta.com/docs/api/resources/system_log).","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P0Z000007eJOWUA2"},"Id":"a1P0Z000007eJOWUA2","CreatedDate":"2018-04-23T17:44:08.000+0000","IncidentId__c":"a9C0Z000000TNelUAG","UpdateLog__c":"Okta continues to investigate an issue with the /api/v1/events endpoint in all cells. To access system logs, please utilize the [System Log API Endpoint](https://developer.okta.com/docs/api/resources/system_log). The next update will take place at 11:45am PT or as soon as additional details become available.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P0Z000007eJPPUA2"},"Id":"a1P0Z000007eJPPUA2","CreatedDate":"2018-04-23T18:43:56.000+0000","IncidentId__c":"a9C0Z000000TNelUAG","UpdateLog__c":"Okta continues to investigate an issue with the /api/v1/events endpoint in all cells. To access system logs, please utilize the [System Log API Endpoint](https://developer.okta.com/docs/api/resources/system_log). The next update will take place at 12:45pm PT or as soon as additional details become available.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P0Z000007eJPoUAM"},"Id":"a1P0Z000007eJPoUAM","CreatedDate":"2018-04-23T19:06:35.000+0000","IncidentId__c":"a9C0Z000000TNelUAG","UpdateLog__c":"Okta is actively mitigating an issue with the /api/v1/events endpoint in all cells. To access system logs, please utilize the [System Log API Endpoint](https://developer.okta.com/docs/api/resources/system_log). The next update will take place at 1:00pm PT or as soon as additional details become available.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P0Z000007eJQ8UAM"},"Id":"a1P0Z000007eJQ8UAM","CreatedDate":"2018-04-23T19:54:35.000+0000","IncidentId__c":"a9C0Z000000TNelUAG","UpdateLog__c":"Okta continues to mitigate an issue with the /api/v1/events endpoint in all cells. To access system logs, please utilize the [System Log API Endpoint](https://developer.okta.com/docs/api/resources/system_log). The next update will take place at 2:00pm PT or as soon as additional details become available.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P0Z000007eJQhUAM"},"Id":"a1P0Z000007eJQhUAM","CreatedDate":"2018-04-23T21:01:13.000+0000","IncidentId__c":"a9C0Z000000TNelUAG","UpdateLog__c":"Okta continues to mitigate an issue with the /api/v1/events endpoint in all cells. To access system logs, please utilize the [System Log API Endpoint](https://developer.okta.com/docs/api/resources/system_log). The next update will take place at 3:00pm PT or as soon as additional details become available.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P0Z000007eJUoUAM"},"Id":"a1P0Z000007eJUoUAM","CreatedDate":"2018-04-24T15:17:52.000+0000","IncidentId__c":"a9C0Z000000TNeqUAG","UpdateLog__c":"An issue impacting authentication for a subset of end users in US Cell 2 is being investigated.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P0Z000007eJVNUA2"},"Id":"a1P0Z000007eJVNUA2","CreatedDate":"2018-04-24T15:55:38.000+0000","IncidentId__c":"a9C0Z000000TNeqUAG","UpdateLog__c":"An issue impacting authentication for a subset of end users in US Cell 2 was addressed. Additional root cause information will provided when available.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P0Z000007eJbWUAU"},"Id":"a1P0Z000007eJbWUAU","CreatedDate":"2018-04-25T02:43:56.000+0000","IncidentId__c":"a9C0Z000000TNeqUAG","UpdateLog__c":"Root Cause Analysis:\r\n\r\nOn April 24th, at 7:53am PDT, Okta experienced a broad service disruption to US Cell 2. End users in US Cell 2 attempting to reach Okta received HTTP 500 errors, and customer integrations attempting to reach Okta API endpoints in US Cell 2 encountered elevated response times and errors. Operational mitigation steps commenced immediately at 7:53am. The greatest frequency of errors was a 14 minute period between 7:53am and 8:07am. The success rate began to gradually improve from 8:07am, and by 8:25am approximately 80% of requests were successfully returned. The cell was fully restored by 8:48am. \r\n\r\nRoot cause was traced to a combination of a procedural error and an automation bug while security patching infrastructure within US Cell 2. The procedural error targeted the security patching to a high usage cache cluster. The automation used to apply the patch disrupted the cluster’s ability to failover across nodes. \r\nTo prevent recurrence of this type of issue in the future, Engineering has immediately implemented additional safeguards to review and confirm operational patching procedures. Additionally, Engineering has prioritized a deep technical assessment to strengthen the patching automation for cache clusters.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P0Z000007eNdTUAU"},"Id":"a1P0Z000007eNdTUAU","CreatedDate":"2018-05-17T19:24:28.000+0000","IncidentId__c":"a9C0Z000000TNgDUAW","UpdateLog__c":"Okta is actively working to mitigate the issues impacting authentication and user interactive errors in US Cell 6. As of 12:19pm PDT, error rates have returned to 0%. We are continuing to monitor and will post updates as we continue to address the root cause.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P2A00000G0UoBUAV"},"Id":"a1P2A00000G0UoBUAV","CreatedDate":"2018-03-07T18:20:10.000+0000","IncidentId__c":"a9C2A000000PBikUAG","UpdateLog__c":"We are working to revert the change impacting API Access Management in our US Preview Cell. If you need immediate assistance in resolving the issue, please contact Okta Customer Support.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P2A00000G0UoaUAF"},"Id":"a1P2A00000G0UoaUAF","CreatedDate":"2018-03-07T19:04:09.000+0000","IncidentId__c":"a9C2A000000PBikUAG","UpdateLog__c":"The issue impacting API Access Management in US Preview cell has been resolved for the majority of customers. Work is on-going to fully restore the feature for all remaining customer tenants.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P2A00000G0UouUAF"},"Id":"a1P2A00000G0UouUAF","CreatedDate":"2018-03-07T19:35:41.000+0000","IncidentId__c":"a9C2A000000PBikUAG","UpdateLog__c":"The issue impacting API Access Management in US Preview cell has been resolved for the majority of customers. Work is on-going to fully restore the feature for all remaining customer tenants. Customers experiencing any issues related to API Access Management should contact Support.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P2A00000G0UozUAF"},"Id":"a1P2A00000G0UozUAF","CreatedDate":"2018-03-07T20:13:43.000+0000","IncidentId__c":"a9C2A000000PBikUAG","UpdateLog__c":"The issue impacting API Access Management in US Preview cell has been resolved for the majority of customers. Work is on-going to fully restore the feature for all remaining customer tenants. Customers experiencing any issues related to API Access Management should contact Support.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P2A00000G0UpEUAV"},"Id":"a1P2A00000G0UpEUAV","CreatedDate":"2018-03-07T20:28:30.000+0000","IncidentId__c":"a9C2A000000PBikUAG","UpdateLog__c":"An issue impacting API Access Management for all end-users in US Preview Cell was resolved. Additional root cause information will be provided here as soon as it is available.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P2A00000G0UpTUAV"},"Id":"a1P2A00000G0UpTUAV","CreatedDate":"2018-03-07T20:45:51.000+0000","IncidentId__c":"a9C2A000000PBikUAG","UpdateLog__c":"Okta continues to address issues with the API Access Management feature. Customers experiencing issues are advised to contact support.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P2A00000G0UpsUAF"},"Id":"a1P2A00000G0UpsUAF","CreatedDate":"2018-03-07T21:25:07.000+0000","IncidentId__c":"a9C2A000000PBikUAG","UpdateLog__c":"The issue impacting API Access Management in US Preview cell has been resolved for the majority of customers. Work is on-going to fully restore the feature for all remaining customer tenants. Customers experiencing any issues related to API Access Management should contact Support.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P2A00000G0Uq2UAF"},"Id":"a1P2A00000G0Uq2UAF","CreatedDate":"2018-03-07T21:50:15.000+0000","IncidentId__c":"a9C2A000000PBikUAG","UpdateLog__c":"An issue impacting API Access Management for all end-users in US Preview Cell was resolved. Additional root cause information will be provided here as soon as it is available.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P2A00000G0UqlUAF"},"Id":"a1P2A00000G0UqlUAF","CreatedDate":"2018-03-08T01:50:29.000+0000","IncidentId__c":"a9C2A000000PBipUAG","UpdateLog__c":"An issue impacting system responsiveness in US Cell 6 is being investigated.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P2A00000G0UqvUAF"},"Id":"a1P2A00000G0UqvUAF","CreatedDate":"2018-03-08T02:15:09.000+0000","IncidentId__c":"a9C2A000000PBipUAG","UpdateLog__c":"An issue impacting system availability and authentication in US Cell 6 is being mitigated.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P2A00000G0Ur0UAF"},"Id":"a1P2A00000G0Ur0UAF","CreatedDate":"2018-03-08T02:45:44.000+0000","IncidentId__c":"a9C2A000000PBipUAG","UpdateLog__c":"An issue impacting system availability in US Cell 6 has been resolved. Additional root cause information will be provided here as soon as it is available","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P2A00000G0UyNUAV"},"Id":"a1P2A00000G0UyNUAV","CreatedDate":"2018-03-09T23:40:54.000+0000","IncidentId__c":"a9C2A000000PBipUAG","UpdateLog__c":"Root Cause Analysis:\r\n\r\nOn March 7th, beginning at 5:15pm PST, Okta experienced a service disruption in US Cell 6. End-User, Admin and API requests within Okta tenants within US Cell 6 were slow, unresponsive, or encountered HTTP 5xx errors.\r\n\r\nThe disruption was the result of a memory issue which impacted the cell's Redis cluster which is used to cache application data. The root cause of the memory issue was traced to a combination of high memory usage within the Redis cluster, a mis-configured application server which increased the volume of caching above expected levels, and sub-optimal Redis configuration settings. Because both the primary and secondary Redis tiers were equally impacted by the increased memory demands, the cluster was unable to recover. To stabilize the Redis cluster, additional capacity was added and application servers were recycled to clear hung connections and re-balance load. By 6:08 pm PST, the cell had returned to normal performance levels.\r\n\r\nOkta has completed work to up-size our Redis caching clusters in US Cell 6 and will deploy the same mitigating changes across all remaining cells. Additionally, Okta will conduct exercises to replay this failure sequence within a test environment to determine and subsequently deploy the optimal settings for Redis memory management, client connection configuration and health-check thresholds. Additionally, new tooling is being investigated to allow us to flush Redis cache should such a mitigation step be needed in the future.\r\n<br><br> \r\nReview the root cause analysis [here](https://support.okta.com/help/Documentation/Knowledge_Article/Root-Cause-Analysis-Service-Disruption-03-07-2017).","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P2A00000G0UymUAF"},"Id":"a1P2A00000G0UymUAF","CreatedDate":"2018-03-10T00:15:52.000+0000","IncidentId__c":"a9C2A000000PBikUAG","UpdateLog__c":"Root Cause Analysis:\r\n\r\nOn March 7th, beginning at 9:35am PST, Okta experienced a feature disruption to API Access Management in the US Preview cell. The disruption was caused while Okta was working to correct the assignment of the API Access Management feature to certain tenants and mistakenly removed the feature from orgs authorized to have it.\r\n\r\nAfter the feature was inadvertently disabled, Okta began work to bulk re-assign the feature to all entitled tenants. However, due the complexity of the process of enabling the API Access Management feature, it took Okta longer than usual to restore the feature to entitled tenants. During the incident, while bulk enablement was processing, Okta Support and Engineering manually enabled the feature for all customers who reported the issue or were detected as being impacted through Okta’s monitoring tools.\r\n\r\nBy 10:30 AM PST, the majority of the impacted tenants had the API Access Management feature reassigned at 1:50 PM PST all entitled customers in the US Preview cell had the feature restored.\r\n\r\nTo prevent recurrence, Okta is adding additional oversight to feature deployment processes and will be hardening our management software to reduce risk of data input errors.\r\n\r\n<br><br> \r\nReview the root cause analysis [here](https://support.okta.com/help/Documentation/Knowledge_Article/Root-Cause-Analysis-API-Access-Management-Service-Disruption-03072018).","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000005V3R7UAK"},"Id":"a1P1Y000005V3R7UAK","CreatedDate":"2018-12-13T14:15:49.000+0000","IncidentId__c":"a9C1Y000000GpJuUAK","UpdateLog__c":"An issue impacting CORS authentication in Okta Preview Cells has been identified and customers using OIDC applications and API AM might be impacted. A fix has been identified and we are working to return the service to normal as quickly as possible.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zJhVUAU"},"Id":"a1P1Y000007zJhVUAU","CreatedDate":"2019-12-02T17:45:09.000+0000","IncidentId__c":"a9C1Y0000004E5WUAU","UpdateLog__c":"Okta is continuing to work to mitigate the job processing delays for a subset of customers in US Cell 7. Queued jobs are processing, but at a reduced rate. We will provide our next update by 10:15am PST.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000005V3swUAC"},"Id":"a1P1Y000005V3swUAC","CreatedDate":"2019-02-08T10:37:44.000+0000","IncidentId__c":"a9C1Y000000GpKsUAK","UpdateLog__c":"On Feb 5th at approximately 4:10pm PDT, Okta began receiving sporadic reports from customers in US Cell 1 and EMEA Cell 1 who were experiencing errors while attempting to provision users to Office 365 or modifying Office365 configuration settings. User authentication was not impacted at any point.\r\nThe root cause was traced to a serialization bug that affected a portion of Office365 application tenants. In particular, those where federation was auto configured using the Okta Admin UI prior to 2016, or manually configured by running the Okta provided Azure AD power-shell cmdlets.\r\n\r\nOnce the root cause was identified, Okta deployed a fix starting at 5:17pm PDT on Feb 5. At 7:42pm PDT, the fix was completely rolled out to all production cells, and functionality was fully restored.\r\n\r\nOkta has updated its proactive monitoring to identify and prevent future occurrences.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zJUvUAM"},"Id":"a1P1Y000007zJUvUAM","CreatedDate":"2019-11-22T16:15:09.000+0000","IncidentId__c":"a9C1Y0000004E4JUAU","UpdateLog__c":"Root Cause Analysis: On Nov 19th at approximately 12:30 PM PDT, customers reported that when opening the Okta Mobile application for Android, end users were intermittently logged out of their account and encountered the following message:\r\n\r\n“Detected possible malicious activity when establishing a secure connection to Okta”.\r\n\r\nUpon investigation, Okta determined that the issue was caused by an update to the preferred authentication algorithm used in TLS by Okta Mobile’s error tracking provider. The new preferred authentication algorithm chosen by the provider became ECDSA, which was not allowed by the Okta Mobile Android application. Upon detecting a change in the encryption algorithm in use, Okta Mobile on Android is designed to present the aforementioned warning message.\r\n\r\nOn Nov 19th at approximately 8:45 PM PDT, Okta released Android Okta Mobile version 4.3.2, for Android versions 6.0 and above, and notified Org administrators through the Admin Console banner that users should update their Okta Mobile application to address the issue. Users running an older version of Android are encouraged to update their device to 6.0 or later and install the latest version of Okta Mobile.\r\n\r\nOkta is implementing additional monitoring and alerting to detect such issues earlier in the future. Additionally, the latest Android Okta Mobile version has expanded the set of authentication algorithms allowed and newer versions will include an improved design to prevent issues like this in the future.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zK4ZUAU"},"Id":"a1P1Y000007zK4ZUAU","CreatedDate":"2020-01-23T01:16:26.000+0000","IncidentId__c":"a9C1Y0000004E8uUAE","UpdateLog__c":"Okta has resolved an issue in US Cell 1 impacting AD/LDAP Agent state reporting. All on-line AD/LDAP Agents are now correctly reporting ACTIVE status. Additionally, some customers where all AD/LDAP Agents were marked NOT_CONNECTED should now see successful imports. Additional root cause information will be provided within 48 hours.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zK5wUAE"},"Id":"a1P1Y000007zK5wUAE","CreatedDate":"2020-01-27T20:17:34.000+0000","IncidentId__c":"a9C1Y0000004E8uUAE","UpdateLog__c":"Root Cause Analysis: On January 22nd at approximately ‪11:39am PST‬ Okta’s monitoring systems detected internal server errors in US Cell 1. The errors resulted in some customers' AD and LDAP agents being reported as NOT_CONNECTED to Okta Admins, even though the agents were functioning normally. Additionally, customers where all AD Agents were marked offline and who had Desktop Single Sign-On (IWA) configured using automatic fail-over may have experienced IWA failures. Okta implemented mitigation steps and the errors subsided at approximately ‪12:06pm PST‬. Okta subsequently corrected agent status for all customers by approximately ‪5:00pm PST‬.\r\n\r\nUpon investigation, Okta determined that a mistake was made when updating references to the database system during the execution of a planned routine database maintenance, This misconfiguration caused a very small number of requests to return an HTTP 500 error code and also resulted in the outdated agent status information.\r\n\r\nOkta has updated run-books and is improving the automation of maintenance verification procedures to prevent future occurrences. Okta is also investigating detection of agent status check job failures to prevent incorrect status indication to administrators.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000005V3qRUAS"},"Id":"a1P1Y000005V3qRUAS","CreatedDate":"2019-02-06T02:10:37.000+0000","IncidentId__c":"a9C1Y000000GpKsUAK","UpdateLog__c":"A fix has been identified where office 365 users are seeing provisioning errors. We are working to return the service to normal as quickly as possible.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000005V3qHUAS"},"Id":"a1P1Y000005V3qHUAS","CreatedDate":"2019-02-06T01:10:05.000+0000","IncidentId__c":"a9C1Y000000GpKsUAK","UpdateLog__c":"Okta is currently experiencing an issue in EU1, OP1, OK1, OK2, OK3, and OK5. Some office 365 users are seeing provisioning errors \"An error occurred while assigning this app\" when assigning users to the application.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000005V3qMUAS"},"Id":"a1P1Y000005V3qMUAS","CreatedDate":"2019-02-06T01:40:53.000+0000","IncidentId__c":"a9C1Y000000GpKsUAK","UpdateLog__c":"A fix has been identified where office 365 users are seeing provisioning errors. We are working to return the service to normal as quickly as possible.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000005V3smUAC"},"Id":"a1P1Y000005V3smUAC","CreatedDate":"2019-02-08T01:34:33.000+0000","IncidentId__c":"a9C1Y000000GpKxUAK","UpdateLog__c":"A fix has been identified where office 365 users are seeing provisioning errors. We are working to return the service to normal as quickly as possible.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000005V3scUAC"},"Id":"a1P1Y000005V3scUAC","CreatedDate":"2019-02-08T01:05:10.000+0000","IncidentId__c":"a9C1Y000000GpKxUAK","UpdateLog__c":"Okta is currently experiencing an issue with Office 365 provisioning. We are actively investigating and will update this message with more information as soon as we have it.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000005qHx9UAE"},"Id":"a1P1Y000005qHx9UAE","CreatedDate":"2019-06-05T01:23:26.000+0000","IncidentId__c":"a9C1Y000000fxWSUAY","UpdateLog__c":"Root Cause: On May 31st, at approximately 12:24 PM PDT Okta received reports from some customers receiving a '400 Bad Request' response while attempting to login with Agentless Desktop SSO using Chrome browsers.\r\n<br><br>\r\nUpon investigation, Okta determined that the issue impacted customers with either incorrect Chrome settings or network configuration (see https://help.okta.com/en/prod/Content/Topics/Directory/Configuring_Agentless_SSO.htm). Okta also determined that the issue had been exposed by the release of a software performance optimization which did not account for this possible browser mis-configuration; prior to the optimization, this mis-configuration would have resulted in users being redirected to the default login page.\r\n<br><br>\r\nGiven the impact and poor user experience, starting on May 31st at 2:30 PM PDT, Okta deployed a change that reverted behavior to the pre-optimization one. The deployment completed to all cells at approximately 5:30PM PDT.\r\n<br><br>\r\nOkta is implementing additional testing and monitoring to detect this condition in the future.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zHjoUAE"},"Id":"a1P1Y000007zHjoUAE","CreatedDate":"2019-09-30T23:20:43.000+0000","IncidentId__c":"a9C1Y0000004DyLUAU","UpdateLog__c":"We’ve narrowed the issue down to Google Chrome version 77 (on both desktop and Android). This is due to a change introduced in Google Chrome version 77. We are working on a workaround to address the impact of slow page load for the Okta Dashboard.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000005V49YUAS"},"Id":"a1P1Y000005V49YUAS","CreatedDate":"2019-02-19T10:25:14.000+0000","IncidentId__c":"a9C1Y000000GpKxUAK","UpdateLog__c":"On Feb 7th at approximately 4:30PM PDT, Okta began receiving reports from customers in US Cell 4 who were experiencing errors while attempting to provision users to Office 365 or while modifying Office 365 configuration settings. User authentication was not impacted at any point.\r\n<br><br>\r\nThe root cause was traced to an inability in some cases to read domain federation settings from Microsoft Office 365. This bug caused a failure to successfully provision users and save Office 365 app settings. Okta also determined that this issue was present in all cells, but only affected a subset of customers.\r\n<br><br>\r\nOnce the root cause was identified, Okta deployed a fix starting at 4:49pm PDT on Feb 7. at approximately 5:30pm PDT, the fix was successfully rolled out to all production cells, and functionality was fully restored.\r\n<br><br>\r\nAdditional monitoring to detect and mitigate this scenario has been rolled out for all environments. Testing will be augmented to prevent future occurrences.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000005qHdsUAE"},"Id":"a1P1Y000005qHdsUAE","CreatedDate":"2019-05-15T23:45:16.000+0000","IncidentId__c":"a9C1Y000000fxVtUAI","UpdateLog__c":"RESOLVED: Okta has identified and resolved an issue resulting in HTTP 400 errors when users were attempting to authenticate under certain CORS use-cases with tenants in US Preview Cell 1. Additional root cause information will provided within 48 hours.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000005qHo7UAE"},"Id":"a1P1Y000005qHo7UAE","CreatedDate":"2019-05-24T13:57:58.000+0000","IncidentId__c":"a9C1Y000000fxVtUAI","UpdateLog__c":"ROOT CAUSE: On May 15th at approximately 1:42PM PDT, customers in US Preview Cell 2 experienced elevated errors when making HTTP OPTIONS-based requests, predominately against the \"/oauth2/v1/userinfo\" and \"/api/v1/sessions/*****\" endpoints. At 3:09PM PDT, Okta began to detect similar errors in US Preview Cell 1.\r\n\r\nThe elevated error rate was determined to be due to a bug within a new framework used to determine request timeout configurations, and specifically impacted the HTTP OPTIONS method. The new timeout framework was disabled to mitigate the issue, and the service was fully restored at approximately 4:35PM PDT for US Preview Cell 1, and at 4:43PM PDT for US Preview Cell 2.\r\n\r\nFollowing this incident, this bug in the new timeout framework has been addressed. Okta will be adding additional test procedures to detect this issue prior to re-enabling the framework.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zJinUAE"},"Id":"a1P1Y000007zJinUAE","CreatedDate":"2019-12-04T21:21:42.000+0000","IncidentId__c":"a9C1Y0000004E65UAE","UpdateLog__c":"Okta has identified the cause of the Workday import triggering users to be placed in No Password state in US Cell 3. We are actively working to mitigate the issue.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zJixUAE"},"Id":"a1P1Y000007zJixUAE","CreatedDate":"2019-12-04T22:22:54.000+0000","IncidentId__c":"a9C1Y0000004E65UAE","UpdateLog__c":"Okta has resolved the issue where Workday imports for some Orgs changed users to a No Password state resulting in those users being unable to authenticate. Root Cause Analysis will be provided here within 2 business days.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zJisUAE"},"Id":"a1P1Y000007zJisUAE","CreatedDate":"2019-12-04T21:37:14.000+0000","IncidentId__c":"a9C1Y0000004E65UAE","UpdateLog__c":"Okta has resolved the issue where Workday imports for some Orgs changed users to a No Password state resulting in those users being unable to authenticate. Affected customers must rerun a Workday import to return affected end-users to Active state.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000005qJ3IUAU"},"Id":"a1P1Y000005qJ3IUAU","CreatedDate":"2019-07-11T18:48:45.000+0000","IncidentId__c":"a9C1Y000000fxgDUAQ","UpdateLog__c":"Resolved: An issue impacting syslog accessibility in all Okta Cells has been identified and fixed. Service has been restored to normal. Additional root cause information will be provided within 48 business hours.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000005qJP9UAM"},"Id":"a1P1Y000005qJP9UAM","CreatedDate":"2019-07-17T06:55:39.000+0000","IncidentId__c":"a9C1Y000000fxgDUAQ","UpdateLog__c":"ROOT CAUSE ANALYSIS: On On July 11th, at approximately 10:47 AM PDT Okta detected System Logs access failures across all cells. \r\n\r\nUpon investigation, Okta determined that the issue was caused by an expired Certificate Revocation List (CRL), which had been updated in the certificate keystore repository, but had not been synced across System Logs storage infrastructure. This resulted in certificate validation failures when users attempted to access System Logs. \r\n\r\nAt 11:06 AM PDT, Okta updated the CRL across all cells. Errors started clearing as the change was applied and caches were progressively refreshed. At approximately 11:28 AM PDT, all errors had been cleared. \r\n\r\nOkta is implementing additional monitoring and alerting to further enhance detection of similar CRL errors in the future. We are also implementing additional process changes and tooling to prevent similar CRL-related certificate validation errors in the future.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000005qJm8UAE"},"Id":"a1P1Y000005qJm8UAE","CreatedDate":"2019-08-19T20:32:00.000+0000","IncidentId__c":"a9C1Y000000fxh6UAA","UpdateLog__c":"Resolved: On Aug 19th at approximately 12:30PM PDT, Okta noted an issue affecting GSuite's OAuth 2.0 endpoints preventing customers who use GSuite for authentication from accessing ASA's web interface or enrolled servers, Google imports, and SSO login to GSuite apps.\r\n\r\nGoogle is reporting the issue is resolved for OAuth 2.0.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000005qK7eUAE"},"Id":"a1P1Y000005qK7eUAE","CreatedDate":"2019-09-10T06:20:40.000+0000","IncidentId__c":"a9C1Y000000fxj2UAA","UpdateLog__c":"ROOT CAUSE: On September 5th, at approximately 12:27 PM PDT Okta detected an issue in the Developer tenant's activation email which impacted developer org creation in US Preview 1 and US Cell 7. \r\n\r\nUpon investigation, Okta determined that a software change to the Developer Tenant Email Template caused the generated emails to have noticeable styling issues and, in some clients, this caused one-time temporary passwords to become unreadable. \r\n\r\nAt approximately 1:49 PM PDT Okta verified a fix and began deploying the change as a patch to the impacted cells. At approximately 2:13 PM PDT the fix was fully deployed and generated emails were sent with visible temporary passwords. \r\n\r\nAdditional testing and monitoring to detect and mitigate this scenario will be rolled out for affected environments to prevent future occurrences.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000005qJosUAE"},"Id":"a1P1Y000005qJosUAE","CreatedDate":"2019-08-22T05:27:49.000+0000","IncidentId__c":"a9C1Y000000fxh6UAA","UpdateLog__c":"Root Cause Analysis: On August 19th, at approximately 10:50 AM PDT Okta detected an increased number of failed Google OAuth 2.0 authentication requests and import failures. In addition to that, customers using GSuite as their main IdP could not authenticate via Advanced Server Access (ASA) to access their infrastructure. \r\n\r\nUpon investigation, Okta determined that Google had experienced a global outage of their OAuth 2.0 endpoints. This prevented external services (ASA, Okta) from authenticating user sessions against Google as an IdP, and no workarounds were provided. \r\n\r\nAt approximately 12:30 PM PDT the issue with Google OAuth 2.0 endpoints was fully resolved and service was restored.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zHjyUAE"},"Id":"a1P1Y000007zHjyUAE","CreatedDate":"2019-10-01T00:43:45.000+0000","IncidentId__c":"a9C1Y0000004DyLUAU","UpdateLog__c":"Our team is continuing to work on the workaround, and we should be able to start deploying it soon.\r\n\r\nNote that this issue impacts Chromium-based browsers, so this issue may also occur if you’re using the Chromium-based preview version of Microsoft Edge. Non-Chromium-Based versions of Microsoft Edge are not impacted.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zIozUAE"},"Id":"a1P1Y000007zIozUAE","CreatedDate":"2019-11-06T18:23:00.000+0000","IncidentId__c":"a9C1Y0000004E2wUAE","UpdateLog__c":"Okta is currently experiencing an issue with Office 365 Sign on for a subset of users. We are actively investigating and a possible fix is being proposed to address this issue.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000005qK3nUAE"},"Id":"a1P1Y000005qK3nUAE","CreatedDate":"2019-09-05T19:56:49.000+0000","IncidentId__c":"a9C1Y000000fxj2UAA","UpdateLog__c":"Okta has identified the cause of the Activation Email regression affecting new Developer tenants and is currently preparing to deploy a fix.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000005qK3xUAE"},"Id":"a1P1Y000005qK3xUAE","CreatedDate":"2019-09-05T21:15:18.000+0000","IncidentId__c":"a9C1Y000000fxj2UAA","UpdateLog__c":"Okta has successfully deployed and verified the fix for the issue with new Developer tenant activation e-mail in US Cell 7. Additional root cause information will provided within 48 hours.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000005qK3sUAE"},"Id":"a1P1Y000005qK3sUAE","CreatedDate":"2019-09-05T20:36:07.000+0000","IncidentId__c":"a9C1Y000000fxj2UAA","UpdateLog__c":"Okta has developed and verified the fix, and is working to deploy the fix to US Cell 7.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zHo0UAE"},"Id":"a1P1Y000007zHo0UAE","CreatedDate":"2019-10-02T01:40:47.000+0000","IncidentId__c":"a9C1Y0000004DyLUAU","UpdateLog__c":"Resolved: The issue with Google Chrome version 77 (on both desktop and Android) which was due to a change introduced in Google Chrome version 77 is now resolved on all Cells. Additional root cause information will be provided within 48 business hours.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zHpmUAE"},"Id":"a1P1Y000007zHpmUAE","CreatedDate":"2019-10-04T02:10:12.000+0000","IncidentId__c":"a9C1Y0000004DyLUAU","UpdateLog__c":"Root Cause Analysis: On Sept 30th, at approximately 3:39PM PDT, customers reported experiencing longer than expected load times for application logos in the Okta end-user dashboard and catalog.\r\n\r\nUpon investigation, Okta determined users in all cells experienced the longer load times after updating their Chrome browser to version 77.0.3865.75 (Official Build) (64-bit). Users of browsers other than Chrome, and users of Chrome earlier than version 77, were not impacted. Okta discovered that Chrome version 77 caused a performance regression with some HTML Canvas operations when the hardware acceleration setting was available and enabled. This behavior is also discussed in the Chromium bug forum as Issue 1001845.\r\n\r\nAt approximately 8:40PM PDT on Sept 30th, Okta started the phased deployment of a workaround, per-cell basis, which was completed on all cells by 10/1/2019 at 6:40 PM PDT.\r\n\r\nDuring the incident, users were advised to workaround the issue by disabling Hardware Acceleration. This workaround is no longer necessary and Hardware Acceleration in Chrome can be re-enabled.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zIpOUAU"},"Id":"a1P1Y000007zIpOUAU","CreatedDate":"2019-11-06T18:49:41.000+0000","IncidentId__c":"a9C1Y0000004E2wUAE","UpdateLog__c":"Okta has successfully tested the fix in OK4 Cell. We are working diligently to apply the fix to all the Okta Cells.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zJVeUAM"},"Id":"a1P1Y000007zJVeUAM","CreatedDate":"2019-11-23T23:51:04.000+0000","IncidentId__c":"a9C1Y0000004E5RUAU","UpdateLog__c":"Okta is continuing to investigate the root cause of the the issue and is taking steps to mitigate the issue. Some Administrators and Users are still experiencing performance degradation, and we are continuing to work to resolve this as quickly as possible.","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zJVtUAM"},"Id":"a1P1Y000007zJVtUAM","CreatedDate":"2019-11-24T02:01:58.000+0000","IncidentId__c":"a9C1Y0000004E5RUAU","UpdateLog__c":"Okta has taken steps to mitigate the issue, and we are seeing improvements in Dashboard and Authentication requests. At this time user provisioning is still disabled as we work through the remaining issues. Further updates will be provided as we continue to progress towards full resolution","CurrencyIsoCode":"USD"},{"attributes":{"type":"IncidentUpdate__c","url":"/services/data/v48.0/sobjects/IncidentUpdate__c/a1P1Y000007zJVjUAM"},"Id":"a1P1Y000007zJVjUAM","CreatedDate":"2019-11-24T00:52:43.000+0000","IncidentId__c":"a9C1Y0000004E5RUAU","UpdateLog__c":"Okta is continuing to investigate the root cause of the the issue and is taking steps to mitigate the issue. The issue is still ongoing and we are continuing to work to resolve this as quickly as possible.","CurrencyIsoCode":"USD"}]