CyberGrants Degraded Performance (Errors)

Issue Summary:

Between July 12th and 13th, 2026, the search functionality within CyberGrants was unavailable. Users were unable to search for grants, organizations or Applications, and some automated background processes e.g. file imports, also failed. The issue began at approximately 3:51 AM ET on July 12th and was resolved by 12:35 PM ET on July 13th following a database restart. No customer data was lost or corrupted.

Root Cause:

CyberGrants uses Oracle as its database platform. A bug within Oracle’s text search engine (a component owned and maintained by Oracle) caused the search service to fail. Specifically, Oracle’s internal process responsible for handling search queries crashed due to a defect in Oracle’s own proprietary software code. Once in this failed state, any operation that relied on text-based search, including grant searches, organization lookups, and certain automated imports, returned errors until the database was restarted. This is a confirmed Oracle product defect, not an issue with the CyberGrants Application code or Bonterra infrastructure. No system changes were made by Bonterra prior to the failure occurring.

Prevention:

• Immediate: We restarted the database to clear the failed state and verified all data remained intact. Any background jobs that failed during the outage were identified and reprocessed successfully.
• Short-term: We have opened a support case with Oracle to obtain a permanent software patch that addresses this defect.
• Long-term: We are implementing additional monitoring to detect any future search service failures within minutes of them occurring, enabling our team to respond before users are significantly impacted. We're also reviewing our database configuration with Oracle to identify any settings that may further reduce the likelihood of this defect being triggered in the first place.