Job:
#OCPBUGS-15430issue5 weeks agoKubeAPIDown alert rename and/or degraded status ASSIGNED
We have many guards making sure that there are always at least two instances of the kube-apiserver. If we ever reach a single kube-apiserver and it causes disruption for the clients, other alerts such as KubeAPIErrorBudgetBurn will fire.
KubeAPIDown is here to make sure that Prometheus and really any client can reach the kube-apiserver, which they can even when there is only one instance of kube-apiserver running. If they can't or that availability is disrupted, `KubeAPIErrorBudgetBurn` will fire.
Comment 23058588 by Marcel Härri at 2023-09-19T06:57:07.949+0000
#OCPBUGS-42620issue7 days agoalert/KubeAPIErrorBudgetBurn should not be at or above info New
Issue 16322985: alert/KubeAPIErrorBudgetBurn should not be at or above info
Description: This is a clone of issue OCPBUGS-42083. The following is the description of the original issue:  --- This test failed 3 times in the last week with the following error:
 {quote}{{{ KubeAPIErrorBudgetBurn was at or above info for at least 2m28s on platformidentification.JobType\{Release:"4.17", FromRelease:"", Platform:"azure", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 1h33m52s, firing for 2m28s: Sep 16 21:20:56.839 - 148s E namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/firing severity/critical ALERTS\{alertname="KubeAPIErrorBudgetBurn", alertstate="firing", long="6h", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="critical", short="30m"}}}}
 {quote}
  
 
 It didn't fail a single time in the previous month on 4.17 nor in the month before we shipped 4.16 so I'm proposing this as a blocker to be investigated.  Below you have the boilerplate Component Readiness text:
 
  
 ----
 Component Readiness has found a potential regression in the following test:
 {code:java}
 [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above info{code}
 Probability of significant regression: 99.04%
 
 Sample (being evaluated) Release: 4.17
 Start Time: 2024-09-10T00:00:00Z
 End Time: 2024-09-17T23:59:59Z
 Success Rate: 85.71%
 Successes: 18
 Failures: 3
 Flakes: 0
 
 Base (historical) Release: 4.16
 Start Time: 2024-05-28T00:00:00Z
 End Time: 2024-06-27T23:59:59Z
 Success Rate: 100.00%
 Successes: 74
 Failures: 0
 Flakes: 0
 
 View the test details report at [https://sippy.dptools.openshift.org/sippy-ng/component_readiness/test_details?Aggregation=none&Architecture=amd64&Architecture=amd64&FeatureSet=default&FeatureSet=default&Installer=ipi&Installer=ipi&Network=ovn&Network=ovn&NetworkAccess=default&Platform=azure&Platform=azure&Scheduler=default&SecurityMode=default&Suite=serial&Suite=serial&Topology=ha&Topology=ha&Upgrade=none&Upgrade=none&baseEndTime=2024-06-27%2023%3A59%3A59&baseRelease=4.16&baseStartTime=2024-05-28%2000%3A00%3A00&capability=Alerts&columnGroupBy=Architecture%2CNetwork%2CPlatform&component=kube-apiserver&confidence=95&dbGroupBy=Platform%2CArchitecture%2CNetwork%2CTopology%2CFeatureSet%2CUpgrade%2CSuite%2CInstaller&environment=amd64%20default%20ipi%20ovn%20azure%20serial%20ha%20none&ignoreDisruption=1&ignoreMissing=0&includeVariant=Architecture%3Aamd64&includeVariant=FeatureSet%3Adefault&includeVariant=Installer%3Aipi&includeVariant=Installer%3Aupi&includeVariant=Owner%3Aeng&includeVariant=Platform%3Aaws&includeVariant=Platform%3Aazure&includeVariant=Platform%3Agcp&includeVariant=Platform%3Ametal&includeVariant=Platform%3Avsphere&includeVariant=Topology%3Aha&minFail=3&pity=5&sampleEndTime=2024-09-17%2023%3A59%3A59&samplePRNumber=&samplePROrg=&samplePRRepo=&sampleRelease=4.17&sampleStartTime=2024-09-10%2000%3A00%3A00&testId=openshift-tests%3Ad6b41cee7afca1c2a0b52f9e6975425f&testName=%5Bbz-kube-apiserver%5D%5Binvariant%5D%20alert%2FKubeAPIErrorBudgetBurn%20should%20not%20be%20at%20or%20above%20info&view=]
Status: New
#OCPBUGS-30267issue7 days ago[IBMCloud] MonitorTests liveness/readiness probe error events repeat MODIFIED
Mar 12 18:52:24.937 - 58s E namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/firing severity/critical ALERTS
{alertname="KubeAPIErrorBudgetBurn", alertstate="firing", long="1h", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="critical", short="5m"}
#OCPBUGS-42083issue7 days agoalert/KubeAPIErrorBudgetBurn should not be at or above info POST
Issue 16291984: alert/KubeAPIErrorBudgetBurn should not be at or above info
Description: This test failed 3 times in the last week with the following error:
 {quote}{{{ KubeAPIErrorBudgetBurn was at or above info for at least 2m28s on platformidentification.JobType\{Release:"4.17", FromRelease:"", Platform:"azure", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 1h33m52s, firing for 2m28s: Sep 16 21:20:56.839 - 148s E namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/firing severity/critical ALERTS\{alertname="KubeAPIErrorBudgetBurn", alertstate="firing", long="6h", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="critical", short="30m"}}}}
 {quote}
  
 
 It didn't fail a single time in the previous month on 4.17 nor in the month before we shipped 4.16 so I'm proposing this as a blocker to be investigated.  Below you have the boilerplate Component Readiness text:
 
  
 ----
 Component Readiness has found a potential regression in the following test:
 {code:java}
 [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above info{code}
 Probability of significant regression: 99.04%
 
 Sample (being evaluated) Release: 4.17
 Start Time: 2024-09-10T00:00:00Z
 End Time: 2024-09-17T23:59:59Z
 Success Rate: 85.71%
 Successes: 18
 Failures: 3
 Flakes: 0
 
 Base (historical) Release: 4.16
 Start Time: 2024-05-28T00:00:00Z
 End Time: 2024-06-27T23:59:59Z
 Success Rate: 100.00%
 Successes: 74
 Failures: 0
 Flakes: 0
 
 View the test details report at [https://sippy.dptools.openshift.org/sippy-ng/component_readiness/test_details?Aggregation=none&Architecture=amd64&Architecture=amd64&FeatureSet=default&FeatureSet=default&Installer=ipi&Installer=ipi&Network=ovn&Network=ovn&NetworkAccess=default&Platform=azure&Platform=azure&Scheduler=default&SecurityMode=default&Suite=serial&Suite=serial&Topology=ha&Topology=ha&Upgrade=none&Upgrade=none&baseEndTime=2024-06-27%2023%3A59%3A59&baseRelease=4.16&baseStartTime=2024-05-28%2000%3A00%3A00&capability=Alerts&columnGroupBy=Architecture%2CNetwork%2CPlatform&component=kube-apiserver&confidence=95&dbGroupBy=Platform%2CArchitecture%2CNetwork%2CTopology%2CFeatureSet%2CUpgrade%2CSuite%2CInstaller&environment=amd64%20default%20ipi%20ovn%20azure%20serial%20ha%20none&ignoreDisruption=1&ignoreMissing=0&includeVariant=Architecture%3Aamd64&includeVariant=FeatureSet%3Adefault&includeVariant=Installer%3Aipi&includeVariant=Installer%3Aupi&includeVariant=Owner%3Aeng&includeVariant=Platform%3Aaws&includeVariant=Platform%3Aazure&includeVariant=Platform%3Agcp&includeVariant=Platform%3Ametal&includeVariant=Platform%3Avsphere&includeVariant=Topology%3Aha&minFail=3&pity=5&sampleEndTime=2024-09-17%2023%3A59%3A59&samplePRNumber=&samplePROrg=&samplePRRepo=&sampleRelease=4.17&sampleStartTime=2024-09-10%2000%3A00%3A00&testId=openshift-tests%3Ad6b41cee7afca1c2a0b52f9e6975425f&testName=%5Bbz-kube-apiserver%5D%5Binvariant%5D%20alert%2FKubeAPIErrorBudgetBurn%20should%20not%20be%20at%20or%20above%20info&view=]
Status: POST
Comment 25602713 by Devan Goodwin at 2024-09-18T11:43:10.370+0000
A few pieces of data, using our [alert dashboard|https://grafana-loki.ci.openshift.org/d/PeFPT7XVk/ci-cluster-alerts?orgId=1&var-platform=aws&var-platform=azure&var-platform=gcp&var-platform=metal&var-platform=vsphere&var-percentile=P95&var-alerts=KubeAPIErrorBudgetBurn&var-namespaces=All&var-releases=4.17&var-upgrade_type=micro&var-upgrade_type=minor&var-upgrade_type=none&var-networks=sdn&var-networks=ovn&var-topologies=ha&var-architectures=amd64&var-min_job_runs=1&var-lookback=3&var-level=Critical&var-level=Warning&var-min_firing_seconds=1], specifically the job run list a little down the page, we can see this alert firing is quite novel, our first 4.17 hit ever detected on Aug 25. Looking back to 4.16 and 4.15, it seems exceedingly rare, I see just one job run on each release, but 4.17 has a couple dozen and they intensify quite a bit around Sep 12-14.
periodic-ci-openshift-release-master-ci-4.17-e2e-azure-ovn-upgrade-out-of-change (all) - 38 runs, 16% failed, 617% of failures match = 97% impact
#1843651858065788928junit21 hours ago
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 4m28s on platformidentification.JobType{Release:"4.17", FromRelease:"4.17", Platform:"azure", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 4m28s, firing for 0s:
Oct 08 14:59:54.860 - 268s  I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
#1843293802605842432junit45 hours ago
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 19m58s on platformidentification.JobType{Release:"4.17", FromRelease:"4.17", Platform:"azure", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 19m58s, firing for 0s:
Oct 07 15:30:38.267 - 1198s I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
#1843409292284137472junit36 hours ago
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 17m26s on platformidentification.JobType{Release:"4.17", FromRelease:"4.17", Platform:"azure", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 17m26s, firing for 0s:
Oct 08 00:10:20.060 - 28s   I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="1d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="2h"}
Oct 08 00:10:20.060 - 1018s I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
#1843522329615798272junit29 hours ago
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 29m24s on platformidentification.JobType{Release:"4.17", FromRelease:"4.17", Platform:"azure", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 29m24s, firing for 0s:
Oct 08 06:42:44.530 - 928s  I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
Oct 08 07:05:14.530 - 538s  I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
Oct 08 07:16:14.530 - 298s  I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
#1843778110093266944junit12 hours ago
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 16m56s on platformidentification.JobType{Release:"4.17", FromRelease:"4.17", Platform:"azure", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 16m56s, firing for 0s:
Oct 08 23:48:28.173 - 988s  I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
Oct 08 23:48:58.173 - 28s   I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="1d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="2h"}
#1843032387835924480junit2 days ago
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 27m40s on platformidentification.JobType{Release:"4.17", FromRelease:"4.17", Platform:"azure", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 27m40s, firing for 0s:
Oct 06 22:30:12.784 - 1242s I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
Oct 06 22:31:32.784 - 418s  I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="1d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="2h"}
#1843032387835924480junit2 days ago
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 13m16s on platformidentification.JobType{Release:"4.17", FromRelease:"4.17", Platform:"azure", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 13m16s, firing for 0s:
#1842922108959592448junit2 days ago
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 4m28s on platformidentification.JobType{Release:"4.17", FromRelease:"4.17", Platform:"azure", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 4m28s, firing for 0s:
Oct 06 14:55:29.637 - 268s  I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
#1842820671776755712junit3 days ago
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 6m58s on platformidentification.JobType{Release:"4.17", FromRelease:"4.17", Platform:"azure", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 6m58s, firing for 0s:
Oct 06 08:31:27.050 - 418s  I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
#1842750325090422784junit3 days ago
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 6m28s on platformidentification.JobType{Release:"4.17", FromRelease:"4.17", Platform:"azure", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 6m28s, firing for 0s:
Oct 06 03:28:33.571 - 388s  I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
#1842676160387878912junit3 days ago
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 1m58s on platformidentification.JobType{Release:"4.17", FromRelease:"4.17", Platform:"azure", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 1m58s, firing for 0s:
Oct 05 22:41:19.973 - 118s  I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
#1842465787307102208junit4 days ago
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 7m2s on platformidentification.JobType{Release:"4.17", FromRelease:"4.17", Platform:"azure", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 7m2s, firing for 0s:
Oct 05 09:09:42.840 - 422s  I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
#1842465787307102208junit4 days ago
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 30m6s on platformidentification.JobType{Release:"4.17", FromRelease:"4.17", Platform:"azure", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 30m6s, firing for 0s:
Oct 05 08:45:42.118 - 572s  I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="1d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="2h"}
Oct 05 08:45:42.118 - 1234s I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
#1842569169912991744junit3 days ago
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 7m58s on platformidentification.JobType{Release:"4.17", FromRelease:"4.17", Platform:"azure", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 7m58s, firing for 0s:
Oct 05 15:39:02.620 - 478s  I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
#1842397071286472704junit4 days ago
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 13m28s on platformidentification.JobType{Release:"4.17", FromRelease:"4.17", Platform:"azure", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 13m28s, firing for 0s:
Oct 05 04:08:13.193 - 808s  I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
#1842252315721994240junit4 days ago
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 28m22s on platformidentification.JobType{Release:"4.17", FromRelease:"4.17", Platform:"azure", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 28m22s, firing for 0s:
Oct 04 18:39:57.518 - 1674s I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
Oct 04 18:41:15.518 - 28s   I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="1d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="2h"}
#1842252315721994240junit4 days ago
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 2m20s on platformidentification.JobType{Release:"4.17", FromRelease:"4.17", Platform:"azure", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 2m20s, firing for 0s:
#1842132685611339776junit5 days ago
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 38m40s on platformidentification.JobType{Release:"4.17", FromRelease:"4.17", Platform:"azure", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 38m40s, firing for 0s:
Oct 04 10:36:30.717 - 1108s I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
Oct 04 10:57:30.717 - 1212s I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
#1842005202748575744junit5 days ago
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 13m56s on platformidentification.JobType{Release:"4.17", FromRelease:"4.17", Platform:"azure", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 13m56s, firing for 0s:
Oct 04 02:11:59.249 - 808s  I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
Oct 04 02:12:59.249 - 28s   I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="1d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="2h"}
#1841894576244133888junit5 days ago
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 5m0s on platformidentification.JobType{Release:"4.17", FromRelease:"4.17", Platform:"azure", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 5m0s, firing for 0s:
Oct 03 20:12:15.749 - 300s  I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
#1841894576244133888junit5 days ago
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 42m0s on platformidentification.JobType{Release:"4.17", FromRelease:"4.17", Platform:"azure", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 42m0s, firing for 0s:
Oct 03 19:20:16.788 - 148s  I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
Oct 03 19:23:16.788 - 58s   I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
Oct 03 19:28:46.788 - 2314s I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
#1841784549768433664junit6 days ago
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 21m4s on platformidentification.JobType{Release:"4.17", FromRelease:"4.17", Platform:"azure", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 21m4s, firing for 0s:
Oct 03 12:39:38.805 - 1264s I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
#1841784549768433664junit6 days ago
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 1h14m56s on platformidentification.JobType{Release:"4.17", FromRelease:"4.17", Platform:"azure", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 1h14m56s, firing for 0s:
Oct 03 11:46:13.458 - 30s   I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="1d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="2h"}
Oct 03 11:46:13.458 - 2942s I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
Oct 03 11:47:15.458 - 508s  I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="1d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="2h"}
Oct 03 11:58:45.458 - 568s  I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="1d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="2h"}
Oct 03 12:17:15.458 - 448s  I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="1d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="2h"}
#1841664601754701824junit6 days ago
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 2m18s on platformidentification.JobType{Release:"4.17", FromRelease:"4.17", Platform:"azure", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 2m18s, firing for 0s:
Oct 03 04:19:00.500 - 138s  I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
#1841664601754701824junit6 days ago
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 18m32s on platformidentification.JobType{Release:"4.17", FromRelease:"4.17", Platform:"azure", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 18m32s, firing for 0s:
Oct 03 03:35:21.218 - 538s  I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
Oct 03 03:54:21.218 - 568s  I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
Oct 03 04:14:51.218 - 6s    I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
#1841534867506991104junit6 days ago
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 28s on platformidentification.JobType{Release:"4.17", FromRelease:"4.17", Platform:"azure", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 28s, firing for 0s:
Oct 02 20:27:40.545 - 28s   I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
#1841534867506991104junit6 days ago
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 26m22s on platformidentification.JobType{Release:"4.17", FromRelease:"4.17", Platform:"azure", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 26m22s, firing for 0s:
Oct 02 19:39:40.739 - 478s  I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
Oct 02 20:00:40.739 - 868s  I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
Oct 02 20:17:40.739 - 236s  I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
#1841377226776383488junit7 days ago
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 26m24s on platformidentification.JobType{Release:"4.17", FromRelease:"4.17", Platform:"azure", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 26m24s, firing for 0s:
Oct 02 08:58:08.639 - 28s   I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
Oct 02 09:04:08.639 - 1108s I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
Oct 02 09:23:38.639 - 448s  I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
#1841187290165022720junit7 days ago
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 28s on platformidentification.JobType{Release:"4.17", FromRelease:"4.17", Platform:"azure", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 28s, firing for 0s:
Oct 01 19:49:17.883 - 28s   I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
#1841024841252081664junit8 days ago
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 3m28s on platformidentification.JobType{Release:"4.17", FromRelease:"4.17", Platform:"azure", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 3m28s, firing for 0s:
Oct 01 09:17:29.193 - 208s  I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
#1840854501402087424junit8 days ago
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 1m58s on platformidentification.JobType{Release:"4.17", FromRelease:"4.17", Platform:"azure", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 1m58s, firing for 0s:
Sep 30 22:15:58.085 - 118s  I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
#1840659789193940992junit9 days ago
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 1m28s on platformidentification.JobType{Release:"4.17", FromRelease:"4.17", Platform:"azure", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 1m28s, firing for 0s:
Sep 30 08:54:56.646 - 88s   I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
#1840446477663473664junit9 days ago
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 12m28s on platformidentification.JobType{Release:"4.17", FromRelease:"4.17", Platform:"azure", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 12m28s, firing for 0s:
Sep 29 19:01:23.038 - 748s  I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
#1839861111583674368junit11 days ago
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 24m24s on platformidentification.JobType{Release:"4.17", FromRelease:"4.17", Platform:"azure", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 24m24s, firing for 0s:
Sep 28 04:19:41.468 - 1346s I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
Sep 28 04:20:55.468 - 118s  I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="1d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="2h"}
#1839861111583674368junit11 days ago
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 2m14s on platformidentification.JobType{Release:"4.17", FromRelease:"4.17", Platform:"azure", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 2m14s, firing for 0s:
#1839781869323816960junit11 days ago
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 12m58s on platformidentification.JobType{Release:"4.17", FromRelease:"4.17", Platform:"azure", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 12m58s, firing for 0s:
Sep 28 00:06:17.520 - 778s  I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
#1839675678815948800junit11 days ago
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 4m58s on platformidentification.JobType{Release:"4.17", FromRelease:"4.17", Platform:"azure", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 4m58s, firing for 0s:
Sep 27 16:44:21.088 - 298s  I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
#1839605077619249152junit12 days ago
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 58s on platformidentification.JobType{Release:"4.17", FromRelease:"4.17", Platform:"azure", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 58s, firing for 0s:
Sep 27 11:23:34.810 - 58s   I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
#1839528622025609216junit12 days ago
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 7m58s on platformidentification.JobType{Release:"4.17", FromRelease:"4.17", Platform:"azure", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 7m58s, firing for 0s:
Sep 27 06:56:06.059 - 478s  I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
#1839454975885840384junit12 days ago
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 3m58s on platformidentification.JobType{Release:"4.17", FromRelease:"4.17", Platform:"azure", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 3m58s, firing for 0s:
Sep 27 01:44:45.996 - 238s  I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
#1839380392444956672junit12 days ago
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 11m58s on platformidentification.JobType{Release:"4.17", FromRelease:"4.17", Platform:"azure", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 11m58s, firing for 0s:
Sep 26 20:04:35.045 - 718s  I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
#1839209103054868480junit13 days ago
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 22m4s on platformidentification.JobType{Release:"4.17", FromRelease:"4.17", Platform:"azure", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 22m4s, firing for 0s:
Sep 26 08:52:18.404 - 88s   I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="1d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="2h"}
Sep 26 08:52:18.404 - 1236s I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
#1839130683335774208junit13 days ago
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 14m28s on platformidentification.JobType{Release:"4.17", FromRelease:"4.17", Platform:"azure", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 14m28s, firing for 0s:
Sep 26 04:10:58.990 - 868s  I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
#1839047071856857088junit13 days ago
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 15m56s on platformidentification.JobType{Release:"4.17", FromRelease:"4.17", Platform:"azure", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 15m56s, firing for 0s:
Sep 25 22:15:37.035 - 928s  I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
Sep 25 22:16:07.035 - 28s   I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="1d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="2h"}
#1838930476069294080junit13 days ago
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 16m28s on platformidentification.JobType{Release:"4.17", FromRelease:"4.17", Platform:"azure", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 16m28s, firing for 0s:
Sep 25 15:02:38.979 - 988s  I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}

Found in 97.37% of runs (616.67% of failures) across 38 total runs and 1 jobs (15.79% failed) in 334ms - clear search | chart view - source code located on github