Issue 17055435: Watch counts still marginally off on vsphere in 4.19
Description: (_Feel free to update this bug's summary to be more specific._)
 Component Readiness has found a potential regression in the following test:
 
 {code:none}[sig-arch][Late] operators should not create watch channels very often [apigroup:apiserver.openshift.io] [Suite:openshift/conformance/parallel]{code}
 
 Significant regression detected.
 Fishers Exact probability of a regression: 99.94%.
 Test pass rate dropped from 100.00% to 91.43%.
 
 Sample (being evaluated) Release: 4.19
 Start Time: 2025-06-04T00:00:00Z
 End Time: 2025-06-11T16:00:00Z
 Success Rate: 91.43%
 Successes: 32
 Failures: 3
 Flakes: 0
 
 Base (historical) Release: 4.15
 Start Time: 2024-01-29T00:00:00Z
 End Time: 2024-02-28T00:00:00Z
 Success Rate: 100.00%
 Successes: 54
 Failures: 0
 Flakes: 0
 
 View the [test details report|https://sippy-auth.dptools.openshift.org/sippy-ng/component_readiness/test_details?Architecture=amd64&Architecture=amd64&FeatureSet=default&FeatureSet=default&Installer=ipi&Installer=ipi&Network=ovn&Network=ovn&Platform=vsphere&Platform=vsphere&Suite=serial&Suite=serial&Topology=ha&Topology=ha&Upgrade=none&Upgrade=none&baseEndTime=2025-02-25%2023%3A59%3A59&baseRelease=4.18&baseStartTime=2025-01-26%2000%3A00%3A00&capability=Other&columnGroupBy=Architecture%2CNetwork%2CPlatform%2CTopology&component=kube-apiserver&confidence=95&dbGroupBy=Platform%2CArchitecture%2CNetwork%2CTopology%2CFeatureSet%2CUpgrade%2CSuite%2CInstaller&environment=Architecture%3Aamd64%20FeatureSet%3Adefault%20Installer%3Aipi%20Network%3Aovn%20Platform%3Avsphere%20Suite%3Aserial%20Topology%3Aha%20Upgrade%3Anone&flakeAsFailure=false&ignoreDisruption=true&ignoreMissing=false&includeMultiReleaseAnalysis=true&includeVariant=Architecture%3Aamd64&includeVariant=CGroupMode%3Av2&includeVariant=ContainerRuntime%3Acrun&includeVariant=ContainerRuntime%3Arunc&includeVariant=FeatureSet%3Adefault&includeVariant=FeatureSet%3Atechpreview&includeVariant=Installer%3Aipi&includeVariant=Installer%3Aupi&includeVariant=JobTier%3Ablocking&includeVariant=JobTier%3Ainforming&includeVariant=JobTier%3Astandard&includeVariant=LayeredProduct%3Anone&includeVariant=Network%3Aovn&includeVariant=Owner%3Aeng&includeVariant=Owner%3Aservice-delivery&includeVariant=Platform%3Aaws&includeVariant=Platform%3Aazure&includeVariant=Platform%3Agcp&includeVariant=Platform%3Ametal&includeVariant=Platform%3Arosa&includeVariant=Platform%3Avsphere&includeVariant=Topology%3Aha&includeVariant=Topology%3Amicroshift&minFail=3&passRateAllTests=0&passRateNewTests=95&pity=5&sampleEndTime=2025-06-11%2023%3A59%3A59&sampleRelease=4.19&sampleStartTime=2025-06-04%2000%3A00%3A00&testBasisRelease=4.15&testId=openshift-tests%3A9ff4e9b171ea809e0d6faf721b2fe737&testName=%5Bsig-arch%5D%5BLate%5D%20operators%20should%20not%20create%20watch%20channels%20very%20often%20%5Bapigroup%3Aapiserver.openshift.io%5D%20%5BSuite%3Aopenshift%2Fconformance%2Fparallel%5D] for additional context.
             
 This is the test that was silently disabled and stopped failing and Forrest has been working steadily to try to get it healthy again with latest updated watch counts. It looks like the last attempt was just very slightly off, failing with a ratio of 1.01. We need to bump a few more slightly. Probably best to scan all test failures of this globally and accomodate.
Status: ASSIGNED

Issue 17015424: Component Readiness Shows Old Test Name For Renamed Tests
Description: See [this example|https://sippy-auth.dptools.openshift.org/sippy-ng/component_readiness/test_details?Aggregation=none&Architecture=amd64&Architecture=amd64&FeatureSet=default&FeatureSet=default&Installer=upi&Installer=upi&LayeredProduct=none&Network=ovn&Network=ovn&NetworkAccess=default&Platform=vsphere&Platform=vsphere&Procedure=none&Scheduler=default&SecurityMode=default&Suite=serial&Suite=serial&Topology=ha&Topology=ha&Upgrade=none&Upgrade=none&baseEndTime=2025-02-25%2023%3A59%3A59&baseRelease=4.18&baseStartTime=2025-01-26%2000%3A00%3A00&capability=Other&columnGroupBy=Architecture%2CNetwork%2CPlatform%2CTopology&component=kube-apiserver&confidence=95&dbGroupBy=Platform%2CArchitecture%2CNetwork%2CTopology%2CFeatureSet%2CUpgrade%2CSuite%2CInstaller&environment=amd64%20default%20upi%20ovn%20vsphere%20serial%20ha%20none&flakeAsFailure=false&ignoreDisruption=true&ignoreMissing=false&includeMultiReleaseAnalysis=true&includeVariant=Architecture%3Aamd64&includeVariant=CGroupMode%3Av2&includeVariant=ContainerRuntime%3Acrun&includeVariant=ContainerRuntime%3Arunc&includeVariant=FeatureSet%3Adefault&includeVariant=FeatureSet%3Atechpreview&includeVariant=Installer%3Aipi&includeVariant=Installer%3Aupi&includeVariant=JobTier%3Ablocking&includeVariant=JobTier%3Ainforming&includeVariant=JobTier%3Astandard&includeVariant=LayeredProduct%3Anone&includeVariant=Network%3Aovn&includeVariant=Owner%3Aeng&includeVariant=Owner%3Aservice-delivery&includeVariant=Platform%3Aaws&includeVariant=Platform%3Aazure&includeVariant=Platform%3Agcp&includeVariant=Platform%3Ametal&includeVariant=Platform%3Arosa&includeVariant=Platform%3Avsphere&includeVariant=Topology%3Aha&includeVariant=Topology%3Amicroshift&minFail=3&passRateAllTests=0&passRateNewTests=95&pity=5&sampleEndTime=2025-05-28%2023%3A59%3A59&sampleRelease=4.19&sampleStartTime=2025-05-21%2000%3A00%3A00&testBasisRelease=4.15&testId=openshift-tests%3A9ff4e9b171ea809e0d6faf721b2fe737&testName=%5Bsig-arch%5D%5BLate%5D%20operators%20should%20not%20create%20watch%20channels%20very%20often%20%5Bapigroup%3Aapiserver.openshift.io%5D%20%5BSuite%3Aopenshift%2Fconformance%2Fparallel%5D]. 
 
 We show: 
 
 [sig-arch][Late] operators should not create watch channels very often [apigroup:apiserver.openshift.io] [Suite:openshift/conformance/parallel]
 
 But if you look to the samples you'll see this test was renamed to 
 
 [sig-arch][Late] operators should not create watch channels very often
 
 The comparison is working, it's just odd we show the old name not the new.
 
 The old name also appears in regressed tests modal.
Status: Refinement

Issue 16880351: operators should not create watch channels very often regression
Description: [TRT-2049|https://issues.redhat.com/browse/TRT-2049] found that the method for collecting watch counts was changed to track only deprecated api calls.  This left a gap in monitoring until it was discovered and the logic moved to audit log analysis monitor tests. 
 {code:none}
 [sig-arch][Late] operators should not create watch channels very often [apigroup:apiserver.openshift.io] [Suite:openshift/conformance/parallel]
 {code}
 is showing [regressed|https://sippy.dptools.openshift.org/sippy-ng/component_readiness/capability?baseEndTime=2025-04-10%2023%3A59%3A59&baseRelease=4.18&baseStartTime=2025-03-11%2000%3A00%3A00&capability=Other&columnGroupBy=Architecture%2CNetwork%2CPlatform%2CTopology&component=kube-apiserver&confidence=95&dbGroupBy=Platform%2CArchitecture%2CNetwork%2CTopology%2CFeatureSet%2CUpgrade%2CSuite%2CInstaller&flakeAsFailure=false&ignoreDisruption=true&ignoreMissing=false&includeMultiReleaseAnalysis=true&includeVariant=Architecture%3Aamd64&includeVariant=CGroupMode%3Av2&includeVariant=ContainerRuntime%3Acrun&includeVariant=ContainerRuntime%3Arunc&includeVariant=FeatureSet%3Adefault&includeVariant=FeatureSet%3Atechpreview&includeVariant=Installer%3Aipi&includeVariant=Installer%3Aupi&includeVariant=JobTier%3Ablocking&includeVariant=JobTier%3Ainforming&includeVariant=JobTier%3Astandard&includeVariant=LayeredProduct%3Anone&includeVariant=Network%3Aovn&includeVariant=Owner%3Aeng&includeVariant=Owner%3Aservice-delivery&includeVariant=Platform%3Aaws&includeVariant=Platform%3Aazure&includeVariant=Platform%3Agcp&includeVariant=Platform%3Ametal&includeVariant=Platform%3Arosa&includeVariant=Platform%3Avsphere&includeVariant=Topology%3Aha&includeVariant=Topology%3Amicroshift&minFail=3&passRateAllTests=0&passRateNewTests=95&pity=5&regressedModal=1&sampleEndTime=2025-04-10%2023%3A59%3A59&sampleRelease=4.19&sampleStartTime=2025-04-03%2000%3A00%3A00] for vshpere and gcp serial jobs currently.
 
 Now that we can collect the data again we need to analyze the current base line, backport the fix to 4.18 and potentially adjust the caps.
Status: MODIFIED

Issue 16466199: ResilientWatchCacheInitialization (Re)enablement - operator watch counts from component readiness
Description: We've seen a high rate of failure for this test since last Thursday.
 
  
 ----
 ({_}Feel free to update this bug's summary to be more specific.{_})
 Component Readiness has found a potential regression in the following test:
 {code:none}
 [sig-arch][Late] operators should not create watch channels very often [apigroup:apiserver.openshift.io] [Suite:openshift/conformance/parallel]{code}
 Extreme regression detected.
 Fishers Exact probability of a regression: 100.00%.
 Test pass rate dropped from 100.00% to 78.33%.
 
 Sample (being evaluated) Release: 4.18
 Start Time: 2024-11-04T00:00:00Z
 End Time: 2024-11-18T23:59:59Z
 Success Rate: 78.33%
 Successes: 47
 Failures: 13
 Flakes: 0
 
 Base (historical) Release: 4.17
 Start Time: 2024-09-01T00:00:00Z
 End Time: 2024-10-01T23:59:59Z
 Success Rate: 100.00%
 Successes: 133
 Failures: 0
 Flakes: 0
 
 View the [test details report|https://sippy.dptools.openshift.org/sippy-ng/component_readiness/test_details?Aggregation=none&Architecture=amd64&Architecture=amd64&FeatureSet=default&FeatureSet=default&Installer=ipi&Installer=ipi&LayeredProduct=none&Network=ovn&Network=ovn&NetworkAccess=default&Platform=metal&Platform=metal&Procedure=none&Scheduler=default&SecurityMode=default&Suite=parallel&Suite=parallel&Topology=ha&Topology=ha&Upgrade=none&Upgrade=none&baseEndTime=2024-10-01%2023%3A59%3A59&baseRelease=4.17&baseStartTime=2024-09-01%2000%3A00%3A00&capability=Other&columnGroupBy=Architecture%2CNetwork%2CPlatform%2CTopology&component=Unknown&confidence=95&dbGroupBy=Platform%2CArchitecture%2CNetwork%2CTopology%2CFeatureSet%2CUpgrade%2CSuite%2CInstaller&environment=amd64%20default%20ipi%20ovn%20metal%20parallel%20ha%20none&ignoreDisruption=1&ignoreMissing=0&includeVariant=Architecture%3Aamd64&includeVariant=CGroupMode%3Av2&includeVariant=ContainerRuntime%3Acrun&includeVariant=ContainerRuntime%3Arunc&includeVariant=FeatureSet%3Adefault&includeVariant=Installer%3Aipi&includeVariant=Installer%3Aupi&includeVariant=Network%3Aovn&includeVariant=Owner%3Aeng&includeVariant=Platform%3Aaws&includeVariant=Platform%3Aazure&includeVariant=Platform%3Agcp&includeVariant=Platform%3Ametal&includeVariant=Platform%3Avsphere&includeVariant=Topology%3Aha&includeVariant=Topology%3Amicroshift&minFail=3&passRateAllTests=0&passRateNewTests=95&pity=5&sampleEndTime=2024-11-18%2023%3A59%3A59&samplePRNumber=&samplePROrg=&samplePRRepo=&sampleRelease=4.18&sampleStartTime=2024-11-04%2000%3A00%3A00&testId=openshift-tests%3A9ff4e9b171ea809e0d6faf721b2fe737&testName=%5Bsig-arch%5D%5BLate%5D%20operators%20should%20not%20create%20watch%20channels%20very%20often%20%5Bapigroup%3Aapiserver.openshift.io%5D%20%5BSuite%3Aopenshift%2Fconformance%2Fparallel%5D&view=] for additional context.
Status: POST

Issue 15116534: [CI] console-operator produces more watch requests than expected
Description: This is a clone of issue OCPBUGS-4550. The following is the description of the original issue:  --- Description of problem:
 {code:none}
 Test located at github.com/openshift/origin/test/extended/apiserver/api_requests.go:449 is failing '[It] operators should not create watch channels very often [Suite:openshift/conformance/parallel]':
 
 "Operator \"console-operator\" produces more watch requests than expected: watchrequestcount=115, upperbound=112, ratio=1.0267857142857142"
 
 
 {code}
 Version-Release number of selected component (if applicable):
 {code:none}
 4.13{code}
 How reproducible:
 
 Found in 0.07% of runs (0.52% of failures) across 36474 total runs and 3989 jobs (13.68% failed) in 400ms
 
 https://search.ci.openshift.org/?search=console-operator%5C%5C%22+produces+more+watch+requests+than+expected&maxAge=48h&context=1&type=bug%2Bissue%2Bjunit&name=&excludeName=&maxMatches=5&maxBytes=20971520&groupBy=job
 
 {code}Steps to Reproduce:
 1. Unknown
 2.
 3.
 {code}
 Actual results:
 {code:none}
 Console operator exceeds watch request limit{code}
 Expected results:
 {code:none}
 Console operator doesn't exceed watch request limit{code}
 Additional info:
 {code:none}
  {code}
Status: ASSIGNED

Comment 26246003 by Devan Goodwin at 2024-12-10T17:00:43.308+0000
Linking to test: [sig-arch][Late] operators should not create watch channels very often [apigroup:apiserver.openshift.io] [Suite:openshift/conformance/parallel]
Comment 26272388 by Ke Wang at 2024-12-13T09:32:10.821+0000

Issue 16884486: operators should not create watch channels very often regression
Description: This is a clone of issue OCPBUGS-55785. The following is the description of the original issue:  --- [TRT-2049|https://issues.redhat.com/browse/TRT-2049] found that the method for collecting watch counts was changed to track only deprecated api calls.  This left a gap in monitoring until it was discovered and the logic moved to audit log analysis monitor tests. 
 {code:none}
 [sig-arch][Late] operators should not create watch channels very often [apigroup:apiserver.openshift.io] [Suite:openshift/conformance/parallel]
 {code}
 is showing [regressed|https://sippy.dptools.openshift.org/sippy-ng/component_readiness/capability?baseEndTime=2025-04-10%2023%3A59%3A59&baseRelease=4.18&baseStartTime=2025-03-11%2000%3A00%3A00&capability=Other&columnGroupBy=Architecture%2CNetwork%2CPlatform%2CTopology&component=kube-apiserver&confidence=95&dbGroupBy=Platform%2CArchitecture%2CNetwork%2CTopology%2CFeatureSet%2CUpgrade%2CSuite%2CInstaller&flakeAsFailure=false&ignoreDisruption=true&ignoreMissing=false&includeMultiReleaseAnalysis=true&includeVariant=Architecture%3Aamd64&includeVariant=CGroupMode%3Av2&includeVariant=ContainerRuntime%3Acrun&includeVariant=ContainerRuntime%3Arunc&includeVariant=FeatureSet%3Adefault&includeVariant=FeatureSet%3Atechpreview&includeVariant=Installer%3Aipi&includeVariant=Installer%3Aupi&includeVariant=JobTier%3Ablocking&includeVariant=JobTier%3Ainforming&includeVariant=JobTier%3Astandard&includeVariant=LayeredProduct%3Anone&includeVariant=Network%3Aovn&includeVariant=Owner%3Aeng&includeVariant=Owner%3Aservice-delivery&includeVariant=Platform%3Aaws&includeVariant=Platform%3Aazure&includeVariant=Platform%3Agcp&includeVariant=Platform%3Ametal&includeVariant=Platform%3Arosa&includeVariant=Platform%3Avsphere&includeVariant=Topology%3Aha&includeVariant=Topology%3Amicroshift&minFail=3&passRateAllTests=0&passRateNewTests=95&pity=5&regressedModal=1&sampleEndTime=2025-04-10%2023%3A59%3A59&sampleRelease=4.19&sampleStartTime=2025-04-03%2000%3A00%3A00] for vshpere and gcp serial jobs currently.
 
 Now that we can collect the data again we need to analyze the current base line, backport the fix to 4.18 and potentially adjust the caps.
Status: MODIFIED