| aggregated-gcp-ovn-upgrade-5.0-micro-release-openshift-release-analysis-aggregator (all) - 12 runs, 50% failed, 50% of failures match = 25% impact | |||
| #2115219726266273792 | junit | 2 hours ago | |
# openshift-tests.[Monitor:legacy-etcd-invariants][bz-etcd] pathological event should not see excessive RequiredInstallerResourcesMissing secrets name: '[Monitor:legacy-etcd-invariants][bz-etcd] pathological event should not see excessive RequiredInstallerResourcesMissing secrets' testsuitename: openshift-tests | |||
| #2115219726266273792 | junit | 2 hours ago | |
# openshift-tests-upgrade.[Monitor:legacy-etcd-invariants][bz-etcd] pathological event should not see excessive RequiredInstallerResourcesMissing secrets name: '[Monitor:legacy-etcd-invariants][bz-etcd] pathological event should not see excessive RequiredInstallerResourcesMissing secrets' testsuitename: openshift-tests-upgrade | |||
| #2115159055721828352 | junit | 3 hours ago | |
# openshift-tests.[Monitor:legacy-etcd-invariants][bz-etcd] pathological event should not see excessive RequiredInstallerResourcesMissing secrets name: '[Monitor:legacy-etcd-invariants][bz-etcd] pathological event should not see excessive RequiredInstallerResourcesMissing secrets' testsuitename: openshift-tests | |||
| #2115159055721828352 | junit | 3 hours ago | |
# openshift-tests-upgrade.[Monitor:legacy-etcd-invariants][bz-etcd] pathological event should not see excessive RequiredInstallerResourcesMissing secrets name: '[Monitor:legacy-etcd-invariants][bz-etcd] pathological event should not see excessive RequiredInstallerResourcesMissing secrets' testsuitename: openshift-tests-upgrade | |||
| #2115216256259854336 | junit | 2 hours ago | |
# openshift-tests.[Monitor:legacy-etcd-invariants][bz-etcd] pathological event should not see excessive RequiredInstallerResourcesMissing secrets name: '[Monitor:legacy-etcd-invariants][bz-etcd] pathological event should not see excessive RequiredInstallerResourcesMissing secrets' testsuitename: openshift-tests | |||
| #2115216256259854336 | junit | 2 hours ago | |
# openshift-tests-upgrade.[Monitor:legacy-etcd-invariants][bz-etcd] pathological event should not see excessive RequiredInstallerResourcesMissing secrets name: '[Monitor:legacy-etcd-invariants][bz-etcd] pathological event should not see excessive RequiredInstallerResourcesMissing secrets' testsuitename: openshift-tests-upgrade | |||
| aggregator-periodic-ci-openshift-release-main-ci-5.0-e2e-gcp-ovn-upgrade (all) - 1 runs, 100% failed, 100% of failures match = 100% impact | |||
| #2115141946090459136 | junit | 3 hours ago | |
# openshift-tests.[Monitor:legacy-etcd-invariants][bz-etcd] pathological event should not see excessive RequiredInstallerResourcesMissing secrets name: '[Monitor:legacy-etcd-invariants][bz-etcd] pathological event should not see excessive RequiredInstallerResourcesMissing secrets' testsuitename: openshift-tests | |||
| #2115141946090459136 | junit | 3 hours ago | |
# openshift-tests-upgrade.[Monitor:legacy-etcd-invariants][bz-etcd] pathological event should not see excessive RequiredInstallerResourcesMissing secrets name: '[Monitor:legacy-etcd-invariants][bz-etcd] pathological event should not see excessive RequiredInstallerResourcesMissing secrets' testsuitename: openshift-tests-upgrade | |||
| periodic-ci-openshift-release-main-okd-scos-4.13-e2e-gcp-ovn-upgrade (all) - 1 runs, 100% failed, 100% of failures match = 100% impact | |||
| #2115129262733266944 | junit | 3 hours ago | |
event happened 186 times, something is wrong: ns/openshift-config-operator deployment/openshift-config-operator hmsg/44b18dd3c0 - pathological/true reason/FeatureGateUpdate FeatureSet="OKD", Version="4.22.0-okd-scos.10", Enabled="AWSServiceLBNetworkSecurityGroup,AzureClusterHostedDNSInstall,AzureWorkloadIdentity,BootImageSkewEnforcement,BuildCSIVolumes,ConsolePluginContentSecurityPolicy,DualReplica,EVPN,EventTTL,ExternalOIDC,ExternalOIDCWithUIDAndExtraClaimMappings,GatewayAPIWithoutOLM,ImageStreamImportMode,InsightsConfig,InsightsOnDemandDataGather,KMSv1,ManagedBootImagesCPMS,MetricsCollectionProfiles,MutableCSINodeAllocatableCount,NewOLM,NewOLMWebhookProviderOpenshiftServiceCA,NoRegistryClusterInstall,RouteExternalCertificate,ServiceAccountTokenNodeBinding,SigstoreImageVerification,SigstoreImageVerificationPKI,StoragePerformantSecurityPolicy,UpgradeStatus,UserNamespacesPodSecurityStandards,UserNamespacesSupport,VSphereHostVMGroupZonal,VSphereMixedNodeEnv,VSphereMultiDisk,VSphereMultiNetworks", Disabled="AWSClusterHostedDNS,AWSClusterHostedDNSInstall,AWSDedicatedHosts,AWSDualStackInstall,AWSEuropeanSovereignCloudInstall,AdditionalStorageConfig,AutomatedEtcdBackup,AzureDedicatedHosts,AzureDualStackInstall,AzureMultiDisk,BootcNodeManagement,CBORServingAndStorage,CRDCompatibilityRequirementOperator,CRIOCredentialProviderConfig,ClientsAllowCBOR,ClientsPreferCBOR,ClusterAPIComputeInstall,ClusterAPIControlPlaneInstall,ClusterAPIInstall,ClusterAPIInstallIBMCloud,ClusterAPIMachineManagement,ClusterAPIMachineManagementAWS,ClusterAPIMachineManagementAzure,ClusterAPIMachineManagementBareMetal,ClusterAPIMachineManagementGCP,ClusterAPIMachineManagementOpenStack,ClusterAPIMachineManagementPowerVS,ClusterAPIMachineManagementVSphere,ClusterMonitoringConfig,ClusterUpdateAcceptRisks,ClusterUpdatePreflight,ClusterVersionOperatorConfiguration,ConfidentialCluster,ConfigurablePKI,DNSNameResolver,DRAPartitionableDevices,DyanmicServiceEndpointIBMCloud,EtcdBackendQuota,EventedPLEG,Example,Example2,ExternalOIDCExternalClaimsSourcing,ExternalOIDCWithUpstreamParity,ExternalSnapshotMetadata,GCPCustomAPIEndpoints,GCPCustomAPIEndpointsInstall,GCPDualStackInstall,GCPSovereignCloudInstall,HyperShiftOnlyDynamicResourceAllocation,ImageModeStatusReporting,IngressControllerDynamicConfigurationManager,IrreconcilableMachineConfig,KMSEncryption,KMSEncryptionProvider,MachineAPIMigration,MachineAPIMigrationAWS,MachineAPIMigrationOpenStack,MachineAPIMigrationVSphere,MachineAPIOperatorDisableMachineHealthCheckController,MaxUnavailableStatefulSet,MinimumKubeletVersion,MixedCPUsAllocation,MultiArchInstallAzure,MultiDiskSetup,MutatingAdmissionPolicy,NetworkConnect,NewOLMBoxCutterRuntime,NewOLMCatalogdAPIV1Metas,NewOLMConfigAPI,NewOLMOwnSingleNamespace,NewOLMPreflightPermissionChecks,NoOverlayMode,NutanixMultiSubnets,OSStreams,OVNObservability,OnPremDNSRecords,OpenShiftPodSecurityAdmission,ProvisioningRequestAvailable,SELinuxMount,ShortCertRotation,SignatureStores,TLSAdherence,VSphereConfigurableMaxAllowedBlockVolumesPerNode,VSphereMultiVCenterDay2,VolumeGroupSnapshot" From: 20:54:58Z To: 20:54:59Z result=reject event happened 36 times, something is wrong: ns/openshift-etcd-operator deployment/etcd-operator hmsg/a1dd45cb26 - pathological/true reason/RequiredInstallerResourcesMissing configmaps: etcd-metrics-proxy-client-ca,etcd-metrics-proxy-serving-ca, configmaps: etcd-metrics-proxy-client-ca-8,etcd-metrics-proxy-serving-ca-8,etcd-peer-client-ca-8,etcd-serving-ca-8 From: 19:55:21Z To: 19:55:22Z result=reject event happened 56 times, something is wrong: ns/openshift-kube-apiserver-operator deployment/kube-apiserver-operator hmsg/8e31e738d9 - pathological/true reason/OperatorStatusChanged Status for clusteroperator/kube-apiserver changed: Degraded message changed from "ConfigObservationDegraded: could not determine if cloud provider is external from featuregate: .spec.featureSet \"OKD\" not found\nConfigObservationDegraded: .spec.featureSet %!q(*v1.FeatureGateEnabledDisabled=<nil>) not found\nStaticPodsDegraded: pod/kube-apiserver-ci-op-7j7nzjp3-fb710-vkd9m-master-0 container \"kube-apiserver\" is waiting: CrashLoopBackOff: back-off 5m0s restarting failed container=kube-apiserver pod=kube-apiserver-ci-op-7j7nzjp3-fb710-vkd9m-master-0_openshift-kube-apiserver(820154c5131260ce2e088579aef5082d)" to "ConfigObservationDegraded: .spec.featureSet %!q(*v1.FeatureGateEnabledDisabled=<nil>) not found\nConfigObservationDegraded: could not determine if cloud provider is external from featuregate: .spec.featureSet \"OKD\" not found\nStaticPodsDegraded: pod/kube-apiserver-ci-op-7j7nzjp3-fb710-vkd9m-master-0 container \"kube-apiserver\" is waiting: CrashLoopBackOff: back-off 5m0s restarting failed container=kube-apiserver pod=kube-apiserver-ci-op-7j7nzjp3-fb710-vkd9m-master-0_openshift-kube-apiserver(820154c5131260ce2e088579aef5082d)" From: 19:55:17Z To: 19:55:18Z result=reject | |||
| #2115129262733266944 | junit | 3 hours ago | |
event happened 175 times, something is wrong: ns/openshift-config-operator deployment/openshift-config-operator hmsg/44b18dd3c0 - pathological/true reason/FeatureGateUpdate FeatureSet="OKD", Version="4.22.0-okd-scos.10", Enabled="AWSServiceLBNetworkSecurityGroup,AzureClusterHostedDNSInstall,AzureWorkloadIdentity,BootImageSkewEnforcement,BuildCSIVolumes,ConsolePluginContentSecurityPolicy,DualReplica,EVPN,EventTTL,ExternalOIDC,ExternalOIDCWithUIDAndExtraClaimMappings,GatewayAPIWithoutOLM,ImageStreamImportMode,InsightsConfig,InsightsOnDemandDataGather,KMSv1,ManagedBootImagesCPMS,MetricsCollectionProfiles,MutableCSINodeAllocatableCount,NewOLM,NewOLMWebhookProviderOpenshiftServiceCA,NoRegistryClusterInstall,RouteExternalCertificate,ServiceAccountTokenNodeBinding,SigstoreImageVerification,SigstoreImageVerificationPKI,StoragePerformantSecurityPolicy,UpgradeStatus,UserNamespacesPodSecurityStandards,UserNamespacesSupport,VSphereHostVMGroupZonal,VSphereMixedNodeEnv,VSphereMultiDisk,VSphereMultiNetworks", Disabled="AWSClusterHostedDNS,AWSClusterHostedDNSInstall,AWSDedicatedHosts,AWSDualStackInstall,AWSEuropeanSovereignCloudInstall,AdditionalStorageConfig,AutomatedEtcdBackup,AzureDedicatedHosts,AzureDualStackInstall,AzureMultiDisk,BootcNodeManagement,CBORServingAndStorage,CRDCompatibilityRequirementOperator,CRIOCredentialProviderConfig,ClientsAllowCBOR,ClientsPreferCBOR,ClusterAPIComputeInstall,ClusterAPIControlPlaneInstall,ClusterAPIInstall,ClusterAPIInstallIBMCloud,ClusterAPIMachineManagement,ClusterAPIMachineManagementAWS,ClusterAPIMachineManagementAzure,ClusterAPIMachineManagementBareMetal,ClusterAPIMachineManagementGCP,ClusterAPIMachineManagementOpenStack,ClusterAPIMachineManagementPowerVS,ClusterAPIMachineManagementVSphere,ClusterMonitoringConfig,ClusterUpdateAcceptRisks,ClusterUpdatePreflight,ClusterVersionOperatorConfiguration,ConfidentialCluster,ConfigurablePKI,DNSNameResolver,DRAPartitionableDevices,DyanmicServiceEndpointIBMCloud,EtcdBackendQuota,EventedPLEG,Example,Example2,ExternalOIDCExternalClaimsSourcing,ExternalOIDCWithUpstreamParity,ExternalSnapshotMetadata,GCPCustomAPIEndpoints,GCPCustomAPIEndpointsInstall,GCPDualStackInstall,GCPSovereignCloudInstall,HyperShiftOnlyDynamicResourceAllocation,ImageModeStatusReporting,IngressControllerDynamicConfigurationManager,IrreconcilableMachineConfig,KMSEncryption,KMSEncryptionProvider,MachineAPIMigration,MachineAPIMigrationAWS,MachineAPIMigrationOpenStack,MachineAPIMigrationVSphere,MachineAPIOperatorDisableMachineHealthCheckController,MaxUnavailableStatefulSet,MinimumKubeletVersion,MixedCPUsAllocation,MultiArchInstallAzure,MultiDiskSetup,MutatingAdmissionPolicy,NetworkConnect,NewOLMBoxCutterRuntime,NewOLMCatalogdAPIV1Metas,NewOLMConfigAPI,NewOLMOwnSingleNamespace,NewOLMPreflightPermissionChecks,NoOverlayMode,NutanixMultiSubnets,OSStreams,OVNObservability,OnPremDNSRecords,OpenShiftPodSecurityAdmission,ProvisioningRequestAvailable,SELinuxMount,ShortCertRotation,SignatureStores,TLSAdherence,VSphereConfigurableMaxAllowedBlockVolumesPerNode,VSphereMultiVCenterDay2,VolumeGroupSnapshot" From: 20:48:20Z To: 20:48:21Z result=reject event happened 58 times, something is wrong: ns/openshift-etcd-operator deployment/etcd-operator hmsg/a1dd45cb26 - pathological/true reason/RequiredInstallerResourcesMissing configmaps: etcd-metrics-proxy-client-ca,etcd-metrics-proxy-serving-ca, configmaps: etcd-metrics-proxy-client-ca-8,etcd-metrics-proxy-serving-ca-8,etcd-peer-client-ca-8,etcd-serving-ca-8 From: 20:55:13Z To: 20:55:14Z result=reject event happened 971 times, something is wrong: ns/openshift-kube-apiserver-operator deployment/kube-apiserver-operator hmsg/bfd1bbeeb0 - pathological/true reason/ObserveCloudProviderNames Could not determine external cloud provider state: could not determine if cloud provider is external from featuregate: .spec.featureSet "OKD" not found From: 19:46:20Z To: 19:46:21Z result=reject | |||
| #2115129262733266944 | junit | 3 hours ago | |
event happened 136 times, something is wrong: ns/openshift-kube-apiserver-operator deployment/kube-apiserver-operator hmsg/0f0a2a7cd7 - pathological/true reason/OperatorStatusChanged Status for clusteroperator/kube-apiserver changed: Degraded message changed from "ConfigObservationDegraded: could not determine if cloud provider is external from featuregate: .spec.featureSet \"OKD\" not found\nConfigObservationDegraded: .spec.featureSet %!q(*v1.FeatureGateEnabledDisabled=<nil>) not found" to "ConfigObservationDegraded: .spec.featureSet %!q(*v1.FeatureGateEnabledDisabled=<nil>) not found\nConfigObservationDegraded: could not determine if cloud provider is external from featuregate: .spec.featureSet \"OKD\" not found" From: 20:12:17Z To: 20:12:18Z result=reject event happened 59 times, something is wrong: ns/openshift-etcd-operator deployment/etcd-operator hmsg/a1dd45cb26 - pathological/true reason/RequiredInstallerResourcesMissing configmaps: etcd-metrics-proxy-client-ca,etcd-metrics-proxy-serving-ca, configmaps: etcd-metrics-proxy-client-ca-8,etcd-metrics-proxy-serving-ca-8,etcd-peer-client-ca-8,etcd-serving-ca-8 From: 21:02:01Z To: 21:02:02Z result=reject event happened 241 times, something is wrong: ns/openshift-config-operator deployment/openshift-config-operator hmsg/44b18dd3c0 - pathological/true reason/FeatureGateUpdate FeatureSet="OKD", Version="4.22.0-okd-scos.10", Enabled="AWSServiceLBNetworkSecurityGroup,AzureClusterHostedDNSInstall,AzureWorkloadIdentity,BootImageSkewEnforcement,BuildCSIVolumes,ConsolePluginContentSecurityPolicy,DualReplica,EVPN,EventTTL,ExternalOIDC,ExternalOIDCWithUIDAndExtraClaimMappings,GatewayAPIWithoutOLM,ImageStreamImportMode,InsightsConfig,InsightsOnDemandDataGather,KMSv1,ManagedBootImagesCPMS,MetricsCollectionProfiles,MutableCSINodeAllocatableCount,NewOLM,NewOLMWebhookProviderOpenshiftServiceCA,NoRegistryClusterInstall,RouteExternalCertificate,ServiceAccountTokenNodeBinding,SigstoreImageVerification,SigstoreImageVerificationPKI,StoragePerformantSecurityPolicy,UpgradeStatus,UserNamespacesPodSecurityStandards,UserNamespacesSupport,VSphereHostVMGroupZonal,VSphereMixedNodeEnv,VSphereMultiDisk,VSphereMultiNetworks", Disabled="AWSClusterHostedDNS,AWSClusterHostedDNSInstall,AWSDedicatedHosts,AWSDualStackInstall,AWSEuropeanSovereignCloudInstall,AdditionalStorageConfig,AutomatedEtcdBackup,AzureDedicatedHosts,AzureDualStackInstall,AzureMultiDisk,BootcNodeManagement,CBORServingAndStorage,CRDCompatibilityRequirementOperator,CRIOCredentialProviderConfig,ClientsAllowCBOR,ClientsPreferCBOR,ClusterAPIComputeInstall,ClusterAPIControlPlaneInstall,ClusterAPIInstall,ClusterAPIInstallIBMCloud,ClusterAPIMachineManagement,ClusterAPIMachineManagementAWS,ClusterAPIMachineManagementAzure,ClusterAPIMachineManagementBareMetal,ClusterAPIMachineManagementGCP,ClusterAPIMachineManagementOpenStack,ClusterAPIMachineManagementPowerVS,ClusterAPIMachineManagementVSphere,ClusterMonitoringConfig,ClusterUpdateAcceptRisks,ClusterUpdatePreflight,ClusterVersionOperatorConfiguration,ConfidentialCluster,ConfigurablePKI,DNSNameResolver,DRAPartitionableDevices,DyanmicServiceEndpointIBMCloud,EtcdBackendQuota,EventedPLEG,Example,Example2,ExternalOIDCExternalClaimsSourcing,ExternalOIDCWithUpstreamParity,ExternalSnapshotMetadata,GCPCustomAPIEndpoints,GCPCustomAPIEndpointsInstall,GCPDualStackInstall,GCPSovereignCloudInstall,HyperShiftOnlyDynamicResourceAllocation,ImageModeStatusReporting,IngressControllerDynamicConfigurationManager,IrreconcilableMachineConfig,KMSEncryption,KMSEncryptionProvider,MachineAPIMigration,MachineAPIMigrationAWS,MachineAPIMigrationOpenStack,MachineAPIMigrationVSphere,MachineAPIOperatorDisableMachineHealthCheckController,MaxUnavailableStatefulSet,MinimumKubeletVersion,MixedCPUsAllocation,MultiArchInstallAzure,MultiDiskSetup,MutatingAdmissionPolicy,NetworkConnect,NewOLMBoxCutterRuntime,NewOLMCatalogdAPIV1Metas,NewOLMConfigAPI,NewOLMOwnSingleNamespace,NewOLMPreflightPermissionChecks,NoOverlayMode,NutanixMultiSubnets,OSStreams,OVNObservability,OnPremDNSRecords,OpenShiftPodSecurityAdmission,ProvisioningRequestAvailable,SELinuxMount,ShortCertRotation,SignatureStores,TLSAdherence,VSphereConfigurableMaxAllowedBlockVolumesPerNode,VSphereMultiVCenterDay2,VolumeGroupSnapshot" From: 21:25:40Z To: 21:25:41Z result=reject | |||
| #2115129262733266944 | junit | 3 hours ago | |
event happened 1684 times, something is wrong: ns/openshift-kube-apiserver-operator deployment/kube-apiserver-operator hmsg/bfd1bbeeb0 - pathological/true reason/ObserveCloudProviderNames Could not determine external cloud provider state: could not determine if cloud provider is external from featuregate: .spec.featureSet "OKD" not found From: 20:29:17Z To: 20:29:18Z result=reject event happened 48 times, something is wrong: ns/openshift-etcd-operator deployment/etcd-operator hmsg/a1dd45cb26 - pathological/true reason/RequiredInstallerResourcesMissing configmaps: etcd-metrics-proxy-client-ca,etcd-metrics-proxy-serving-ca, configmaps: etcd-metrics-proxy-client-ca-8,etcd-metrics-proxy-serving-ca-8,etcd-peer-client-ca-8,etcd-serving-ca-8 From: 20:35:13Z To: 20:35:14Z result=reject event happened 229 times, something is wrong: ns/openshift-kube-apiserver-operator deployment/kube-apiserver-operator hmsg/92cc6bb7a9 - pathological/true reason/OperatorStatusChanged Status for clusteroperator/kube-apiserver changed: Degraded message changed from "ConfigObservationDegraded: .spec.featureSet %!q(*v1.FeatureGateEnabledDisabled=<nil>) not found\nConfigObservationDegraded: could not determine if cloud provider is external from featuregate: .spec.featureSet \"OKD\" not found\nStaticPodsDegraded: pod/kube-apiserver-ci-op-7j7nzjp3-fb710-vkd9m-master-0 container \"kube-apiserver\" is waiting: CrashLoopBackOff: back-off 5m0s restarting failed container=kube-apiserver pod=kube-apiserver-ci-op-7j7nzjp3-fb710-vkd9m-master-0_openshift-kube-apiserver(820154c5131260ce2e088579aef5082d)" to "ConfigObservationDegraded: could not determine if cloud provider is external from featuregate: .spec.featureSet \"OKD\" not found\nConfigObservationDegraded: .spec.featureSet %!q(*v1.FeatureGateEnabledDisabled=<nil>) not found\nStaticPodsDegraded: pod/kube-apiserver-ci-op-7j7nzjp3-fb710-vkd9m-master-0 container \"kube-apiserver\" is waiting: CrashLoopBackOff: back-off 5m0s restarting failed container=kube-apiserver pod=kube-apiserver-ci-op-7j7nzjp3-fb710-vkd9m-master-0_openshift-kube-apiserver(820154c5131260ce2e088579aef5082d)" From: 21:25:16Z To: 21:25:17Z result=reject | |||
| #2115129262733266944 | junit | 3 hours ago | |
event happened 1817 times, something is wrong: ns/openshift-kube-apiserver-operator deployment/kube-apiserver-operator hmsg/bfd1bbeeb0 - pathological/true reason/ObserveCloudProviderNames Could not determine external cloud provider state: could not determine if cloud provider is external from featuregate: .spec.featureSet "OKD" not found From: 20:34:17Z To: 20:34:18Z result=reject event happened 44 times, something is wrong: ns/openshift-etcd-operator deployment/etcd-operator hmsg/a1dd45cb26 - pathological/true reason/RequiredInstallerResourcesMissing configmaps: etcd-metrics-proxy-client-ca,etcd-metrics-proxy-serving-ca, configmaps: etcd-metrics-proxy-client-ca-8,etcd-metrics-proxy-serving-ca-8,etcd-peer-client-ca-8,etcd-serving-ca-8 From: 20:25:13Z To: 20:25:14Z result=reject event happened 1658 times, something is wrong: ns/openshift-kube-apiserver-operator deployment/kube-apiserver-operator hmsg/bfd1bbeeb0 - pathological/true reason/ObserveCloudProviderNames Could not determine external cloud provider state: could not determine if cloud provider is external from featuregate: .spec.featureSet "OKD" not found From: 20:28:43Z To: 20:28:44Z result=reject | |||
| #2115129262733266944 | junit | 3 hours ago | |
event happened 206 times, something is wrong: ns/openshift-kube-apiserver-operator deployment/kube-apiserver-operator hmsg/8e31e738d9 - pathological/true reason/OperatorStatusChanged Status for clusteroperator/kube-apiserver changed: Degraded message changed from "ConfigObservationDegraded: could not determine if cloud provider is external from featuregate: .spec.featureSet \"OKD\" not found\nConfigObservationDegraded: .spec.featureSet %!q(*v1.FeatureGateEnabledDisabled=<nil>) not found\nStaticPodsDegraded: pod/kube-apiserver-ci-op-7j7nzjp3-fb710-vkd9m-master-0 container \"kube-apiserver\" is waiting: CrashLoopBackOff: back-off 5m0s restarting failed container=kube-apiserver pod=kube-apiserver-ci-op-7j7nzjp3-fb710-vkd9m-master-0_openshift-kube-apiserver(820154c5131260ce2e088579aef5082d)" to "ConfigObservationDegraded: .spec.featureSet %!q(*v1.FeatureGateEnabledDisabled=<nil>) not found\nConfigObservationDegraded: could not determine if cloud provider is external from featuregate: .spec.featureSet \"OKD\" not found\nStaticPodsDegraded: pod/kube-apiserver-ci-op-7j7nzjp3-fb710-vkd9m-master-0 container \"kube-apiserver\" is waiting: CrashLoopBackOff: back-off 5m0s restarting failed container=kube-apiserver pod=kube-apiserver-ci-op-7j7nzjp3-fb710-vkd9m-master-0_openshift-kube-apiserver(820154c5131260ce2e088579aef5082d)" From: 21:11:31Z To: 21:11:32Z result=reject event happened 69 times, something is wrong: ns/openshift-etcd-operator deployment/etcd-operator hmsg/a1dd45cb26 - pathological/true reason/RequiredInstallerResourcesMissing configmaps: etcd-metrics-proxy-client-ca,etcd-metrics-proxy-serving-ca, configmaps: etcd-metrics-proxy-client-ca-8,etcd-metrics-proxy-serving-ca-8,etcd-peer-client-ca-8,etcd-serving-ca-8 From: 21:18:41Z To: 21:18:42Z result=reject event happened 2674 times, something is wrong: ns/openshift-kube-apiserver-operator deployment/kube-apiserver-operator hmsg/bfd1bbeeb0 - pathological/true reason/ObserveCloudProviderNames Could not determine external cloud provider state: could not determine if cloud provider is external from featuregate: .spec.featureSet "OKD" not found From: 21:28:16Z To: 21:28:17Z result=reject | |||
| periodic-ci-openshift-release-main-okd-scos-4.13-e2e-aws-ovn-upgrade (all) - 1 runs, 100% failed, 100% of failures match = 100% impact | |||
| #2115135947786948608 | junit | 4 hours ago | |
event happened 1900 times, something is wrong: ns/openshift-kube-apiserver-operator deployment/kube-apiserver-operator hmsg/bfd1bbeeb0 - pathological/true reason/ObserveCloudProviderNames Could not determine external cloud provider state: could not determine if cloud provider is external from featuregate: .spec.featureSet "OKD" not found From: 20:48:29Z To: 20:48:30Z result=reject event happened 98 times, something is wrong: ns/openshift-etcd-operator deployment/etcd-operator hmsg/a1dd45cb26 - pathological/true reason/RequiredInstallerResourcesMissing configmaps: etcd-metrics-proxy-client-ca,etcd-metrics-proxy-serving-ca, configmaps: etcd-metrics-proxy-client-ca-8,etcd-metrics-proxy-serving-ca-8,etcd-peer-client-ca-8,etcd-serving-ca-8 From: 20:48:32Z To: 20:48:33Z result=reject event happened 99 times, something is wrong: ns/openshift-config-operator deployment/openshift-config-operator hmsg/44b18dd3c0 - pathological/true reason/FeatureGateUpdate FeatureSet="OKD", Version="4.22.0-okd-scos.10", Enabled="AWSServiceLBNetworkSecurityGroup,AzureClusterHostedDNSInstall,AzureWorkloadIdentity,BootImageSkewEnforcement,BuildCSIVolumes,ConsolePluginContentSecurityPolicy,DualReplica,EVPN,EventTTL,ExternalOIDC,ExternalOIDCWithUIDAndExtraClaimMappings,GatewayAPIWithoutOLM,ImageStreamImportMode,InsightsConfig,InsightsOnDemandDataGather,KMSv1,ManagedBootImagesCPMS,MetricsCollectionProfiles,MutableCSINodeAllocatableCount,NewOLM,NewOLMWebhookProviderOpenshiftServiceCA,NoRegistryClusterInstall,RouteExternalCertificate,ServiceAccountTokenNodeBinding,SigstoreImageVerification,SigstoreImageVerificationPKI,StoragePerformantSecurityPolicy,UpgradeStatus,UserNamespacesPodSecurityStandards,UserNamespacesSupport,VSphereHostVMGroupZonal,VSphereMixedNodeEnv,VSphereMultiDisk,VSphereMultiNetworks", Disabled="AWSClusterHostedDNS,AWSClusterHostedDNSInstall,AWSDedicatedHosts,AWSDualStackInstall,AWSEuropeanSovereignCloudInstall,AdditionalStorageConfig,AutomatedEtcdBackup,AzureDedicatedHosts,AzureDualStackInstall,AzureMultiDisk,BootcNodeManagement,CBORServingAndStorage,CRDCompatibilityRequirementOperator,CRIOCredentialProviderConfig,ClientsAllowCBOR,ClientsPreferCBOR,ClusterAPIComputeInstall,ClusterAPIControlPlaneInstall,ClusterAPIInstall,ClusterAPIInstallIBMCloud,ClusterAPIMachineManagement,ClusterAPIMachineManagementAWS,ClusterAPIMachineManagementAzure,ClusterAPIMachineManagementBareMetal,ClusterAPIMachineManagementGCP,ClusterAPIMachineManagementOpenStack,ClusterAPIMachineManagementPowerVS,ClusterAPIMachineManagementVSphere,ClusterMonitoringConfig,ClusterUpdateAcceptRisks,ClusterUpdatePreflight,ClusterVersionOperatorConfiguration,ConfidentialCluster,ConfigurablePKI,DNSNameResolver,DRAPartitionableDevices,DyanmicServiceEndpointIBMCloud,EtcdBackendQuota,EventedPLEG,Example,Example2,ExternalOIDCExternalClaimsSourcing,ExternalOIDCWithUpstreamParity,ExternalSnapshotMetadata,GCPCustomAPIEndpoints,GCPCustomAPIEndpointsInstall,GCPDualStackInstall,GCPSovereignCloudInstall,HyperShiftOnlyDynamicResourceAllocation,ImageModeStatusReporting,IngressControllerDynamicConfigurationManager,IrreconcilableMachineConfig,KMSEncryption,KMSEncryptionProvider,MachineAPIMigration,MachineAPIMigrationAWS,MachineAPIMigrationOpenStack,MachineAPIMigrationVSphere,MachineAPIOperatorDisableMachineHealthCheckController,MaxUnavailableStatefulSet,MinimumKubeletVersion,MixedCPUsAllocation,MultiArchInstallAzure,MultiDiskSetup,MutatingAdmissionPolicy,NetworkConnect,NewOLMBoxCutterRuntime,NewOLMCatalogdAPIV1Metas,NewOLMConfigAPI,NewOLMOwnSingleNamespace,NewOLMPreflightPermissionChecks,NoOverlayMode,NutanixMultiSubnets,OSStreams,OVNObservability,OnPremDNSRecords,OpenShiftPodSecurityAdmission,ProvisioningRequestAvailable,SELinuxMount,ShortCertRotation,SignatureStores,TLSAdherence,VSphereConfigurableMaxAllowedBlockVolumesPerNode,VSphereMultiVCenterDay2,VolumeGroupSnapshot" From: 20:08:19Z To: 20:08:20Z result=reject | |||
| #2115135947786948608 | junit | 4 hours ago | |
event happened 59 times, something is wrong: ns/openshift-kube-apiserver-operator deployment/kube-apiserver-operator hmsg/39919a6768 - pathological/true reason/OperatorStatusChanged Status for clusteroperator/kube-apiserver changed: Degraded message changed from "ConfigObservationDegraded: .spec.featureSet %!q(*v1.FeatureGateEnabledDisabled=<nil>) not found\nConfigObservationDegraded: could not determine if cloud provider is external from featuregate: .spec.featureSet \"OKD\" not found" to "ConfigObservationDegraded: could not determine if cloud provider is external from featuregate: .spec.featureSet \"OKD\" not found\nConfigObservationDegraded: .spec.featureSet %!q(*v1.FeatureGateEnabledDisabled=<nil>) not found" From: 19:28:33Z To: 19:28:34Z result=reject event happened 64 times, something is wrong: ns/openshift-etcd-operator deployment/etcd-operator hmsg/a1dd45cb26 - pathological/true reason/RequiredInstallerResourcesMissing configmaps: etcd-metrics-proxy-client-ca,etcd-metrics-proxy-serving-ca, configmaps: etcd-metrics-proxy-client-ca-8,etcd-metrics-proxy-serving-ca-8,etcd-peer-client-ca-8,etcd-serving-ca-8 From: 20:15:12Z To: 20:15:13Z result=reject event happened 36 times, something is wrong: ns/openshift-authentication-operator deployment/authentication-operator hmsg/e65a7b8beb - pathological/true reason/CustomResourceDefinitionUpdated Updated CustomResourceDefinition.apiextensions.k8s.io/rolebindingrestrictions.authorization.openshift.io because it changed From: 20:40:47Z To: 20:40:48Z result=reject | |||
| #2115135947786948608 | junit | 4 hours ago | |
event happened 173 times, something is wrong: ns/openshift-kube-apiserver-operator deployment/kube-apiserver-operator hmsg/0f0a2a7cd7 - pathological/true reason/OperatorStatusChanged Status for clusteroperator/kube-apiserver changed: Degraded message changed from "ConfigObservationDegraded: could not determine if cloud provider is external from featuregate: .spec.featureSet \"OKD\" not found\nConfigObservationDegraded: .spec.featureSet %!q(*v1.FeatureGateEnabledDisabled=<nil>) not found" to "ConfigObservationDegraded: .spec.featureSet %!q(*v1.FeatureGateEnabledDisabled=<nil>) not found\nConfigObservationDegraded: could not determine if cloud provider is external from featuregate: .spec.featureSet \"OKD\" not found" From: 20:12:32Z To: 20:12:33Z result=reject event happened 109 times, something is wrong: ns/openshift-etcd-operator deployment/etcd-operator hmsg/a1dd45cb26 - pathological/true reason/RequiredInstallerResourcesMissing configmaps: etcd-metrics-proxy-client-ca,etcd-metrics-proxy-serving-ca, configmaps: etcd-metrics-proxy-client-ca-8,etcd-metrics-proxy-serving-ca-8,etcd-peer-client-ca-8,etcd-serving-ca-8 From: 20:58:29Z To: 20:58:30Z result=reject event happened 478 times, something is wrong: ns/openshift-kube-apiserver-operator deployment/kube-apiserver-operator hmsg/bfd1bbeeb0 - pathological/true reason/ObserveCloudProviderNames Could not determine external cloud provider state: could not determine if cloud provider is external from featuregate: .spec.featureSet "OKD" not found From: 19:23:34Z To: 19:23:35Z result=reject event happened 30 times, something is wrong: ns/openshift-config-operator deployment/openshift-config-operator hmsg/44b18dd3c0 - pathological/true reason/FeatureGateUpdate FeatureSet="OKD", Version="4.22.0-okd-scos.10", Enabled="AWSServiceLBNetworkSecurityGroup,AzureClusterHostedDNSInstall,AzureWorkloadIdentity,BootImageSkewEnforcement,BuildCSIVolumes,ConsolePluginContentSecurityPolicy,DualReplica,EVPN,EventTTL,ExternalOIDC,ExternalOIDCWithUIDAndExtraClaimMappings,GatewayAPIWithoutOLM,ImageStreamImportMode,InsightsConfig,InsightsOnDemandDataGather,KMSv1,ManagedBootImagesCPMS,MetricsCollectionProfiles,MutableCSINodeAllocatableCount,NewOLM,NewOLMWebhookProviderOpenshiftServiceCA,NoRegistryClusterInstall,RouteExternalCertificate,ServiceAccountTokenNodeBinding,SigstoreImageVerification,SigstoreImageVerificationPKI,StoragePerformantSecurityPolicy,UpgradeStatus,UserNamespacesPodSecurityStandards,UserNamespacesSupport,VSphereHostVMGroupZonal,VSphereMixedNodeEnv,VSphereMultiDisk,VSphereMultiNetworks", Disabled="AWSClusterHostedDNS,AWSClusterHostedDNSInstall,AWSDedicatedHosts,AWSDualStackInstall,AWSEuropeanSovereignCloudInstall,AdditionalStorageConfig,AutomatedEtcdBackup,AzureDedicatedHosts,AzureDualStackInstall,AzureMultiDisk,BootcNodeManagement,CBORServingAndStorage,CRDCompatibilityRequirementOperator,CRIOCredentialProviderConfig,ClientsAllowCBOR,ClientsPreferCBOR,ClusterAPIComputeInstall,ClusterAPIControlPlaneInstall,ClusterAPIInstall,ClusterAPIInstallIBMCloud,ClusterAPIMachineManagement,ClusterAPIMachineManagementAWS,ClusterAPIMachineManagementAzure,ClusterAPIMachineManagementBareMetal,ClusterAPIMachineManagementGCP,ClusterAPIMachineManagementOpenStack,ClusterAPIMachineManagementPowerVS,ClusterAPIMachineManagementVSphere,ClusterMonitoringConfig,ClusterUpdateAcceptRisks,ClusterUpdatePreflight,ClusterVersionOperatorConfiguration,ConfidentialCluster,ConfigurablePKI,DNSNameResolver,DRAPartitionableDevices,DyanmicServiceEndpointIBMCloud,EtcdBackendQuota,EventedPLEG,Example,Example2,ExternalOIDCExternalClaimsSourcing,ExternalOIDCWithUpstreamParity,ExternalSnapshotMetadata,GCPCustomAPIEndpoints,GCPCustomAPIEndpointsInstall,GCPDualStackInstall,GCPSovereignCloudInstall,HyperShiftOnlyDynamicResourceAllocation,ImageModeStatusReporting,IngressControllerDynamicConfigurationManager,IrreconcilableMachineConfig,KMSEncryption,KMSEncryptionProvider,MachineAPIMigration,MachineAPIMigrationAWS,MachineAPIMigrationOpenStack,MachineAPIMigrationVSphere,MachineAPIOperatorDisableMachineHealthCheckController,MaxUnavailableStatefulSet,MinimumKubeletVersion,MixedCPUsAllocation,MultiArchInstallAzure,MultiDiskSetup,MutatingAdmissionPolicy,NetworkConnect,NewOLMBoxCutterRuntime,NewOLMCatalogdAPIV1Metas,NewOLMConfigAPI,NewOLMOwnSingleNamespace,NewOLMPreflightPermissionChecks,NoOverlayMode,NutanixMultiSubnets,OSStreams,OVNObservability,OnPremDNSRecords,OpenShiftPodSecurityAdmission,ProvisioningRequestAvailable,SELinuxMount,ShortCertRotation,SignatureStores,TLSAdherence,VSphereConfigurableMaxAllowedBlockVolumesPerNode,VSphereMultiVCenterDay2,VolumeGroupSnapshot" From: 19:29:00Z To: 19:29:01Z result=reject event happened 35 times, something is wrong: ns/openshift-etcd-operator deployment/etcd-operator hmsg/a1dd45cb26 - pathological/true reason/RequiredInstallerResourcesMissing configmaps: etcd-metrics-proxy-client-ca,etcd-metrics-proxy-serving-ca, configmaps: etcd-metrics-proxy-client-ca-8,etcd-metrics-proxy-serving-ca-8,etcd-peer-client-ca-8,etcd-serving-ca-8 From: 19:38:29Z To: 19:38:30Z result=reject event happened 1811 times, something is wrong: ns/openshift-kube-apiserver-operator deployment/kube-apiserver-operator hmsg/bfd1bbeeb0 - pathological/true reason/ObserveCloudProviderNames Could not determine external cloud provider state: could not determine if cloud provider is external from featuregate: .spec.featureSet "OKD" not found From: 20:40:32Z To: 20:40:33Z result=reject | |||
| #2115135947786948608 | junit | 4 hours ago | |
event happened 86 times, something is wrong: ns/openshift-kube-apiserver-operator deployment/kube-apiserver-operator hmsg/bfd1bbeeb0 - pathological/true reason/ObserveCloudProviderNames Could not determine external cloud provider state: could not determine if cloud provider is external from featuregate: .spec.featureSet "OKD" not found From: 19:08:41Z To: 19:08:42Z result=reject event happened 85 times, something is wrong: ns/openshift-etcd-operator deployment/etcd-operator hmsg/a1dd45cb26 - pathological/true reason/RequiredInstallerResourcesMissing configmaps: etcd-metrics-proxy-client-ca,etcd-metrics-proxy-serving-ca, configmaps: etcd-metrics-proxy-client-ca-8,etcd-metrics-proxy-serving-ca-8,etcd-peer-client-ca-8,etcd-serving-ca-8 From: 20:31:52Z To: 20:31:53Z result=reject event happened 385 times, something is wrong: ns/openshift-kube-apiserver-operator deployment/kube-apiserver-operator hmsg/39919a6768 - pathological/true reason/OperatorStatusChanged Status for clusteroperator/kube-apiserver changed: Degraded message changed from "ConfigObservationDegraded: .spec.featureSet %!q(*v1.FeatureGateEnabledDisabled=<nil>) not found\nConfigObservationDegraded: could not determine if cloud provider is external from featuregate: .spec.featureSet \"OKD\" not found" to "ConfigObservationDegraded: could not determine if cloud provider is external from featuregate: .spec.featureSet \"OKD\" not found\nConfigObservationDegraded: .spec.featureSet %!q(*v1.FeatureGateEnabledDisabled=<nil>) not found" From: 21:35:37Z To: 21:35:38Z result=reject event happened 36 times, something is wrong: ns/openshift-etcd-operator deployment/etcd-operator hmsg/a1dd45cb26 - pathological/true reason/RequiredInstallerResourcesMissing configmaps: etcd-metrics-proxy-client-ca,etcd-metrics-proxy-serving-ca, configmaps: etcd-metrics-proxy-client-ca-8,etcd-metrics-proxy-serving-ca-8,etcd-peer-client-ca-8,etcd-serving-ca-8 From: 19:41:52Z To: 19:41:53Z result=reject event happened 21 times, something is wrong: ns/openshift-kube-apiserver-operator deployment/kube-apiserver-operator hmsg/87fdddf542 - pathological/true reason/OperatorStatusChanged Status for clusteroperator/kube-apiserver changed: Degraded message changed from "ConfigObservationDegraded: could not determine if cloud provider is external from featuregate: .spec.featureSet \"OKD\" not found\nConfigObservationDegraded: .spec.featureSet %!q(*v1.FeatureGateEnabledDisabled=<nil>) not found\nStaticPodsDegraded: pod/kube-apiserver-ip-10-0-18-50.us-west-2.compute.internal container \"kube-apiserver\" is waiting: CrashLoopBackOff: back-off 5m0s restarting failed container=kube-apiserver pod=kube-apiserver-ip-10-0-18-50.us-west-2.compute.internal_openshift-kube-apiserver(217842e646fa4f7c0e1082ad28e63cde)" to "ConfigObservationDegraded: .spec.featureSet %!q(*v1.FeatureGateEnabledDisabled=<nil>) not found\nConfigObservationDegraded: could not determine if cloud provider is external from featuregate: .spec.featureSet \"OKD\" not found\nStaticPodsDegraded: pod/kube-apiserver-ip-10-0-18-50.us-west-2.compute.internal container \"kube-apiserver\" is waiting: CrashLoopBackOff: back-off 5m0s restarting failed container=kube-apiserver pod=kube-apiserver-ip-10-0-18-50.us-west-2.compute.internal_openshift-kube-apiserver(217842e646fa4f7c0e1082ad28e63cde)" From: 20:03:01Z To: 20:03:02Z result=reject | |||
| periodic-ci-openshift-release-main-nightly-4.15-upgrade-from-stable-4.14-e2e-aws-upgrade-ovn-single-node (all) - 1 runs, 100% failed, 100% of failures match = 100% impact | |||
| #2115128263629082624 | junit | 4 hours ago | |
event happened 29 times, something is wrong: namespace/openshift-kube-apiserver-operator deployment/kube-apiserver-operator hmsg/d4d3513dba - reason/RequiredInstallerResourcesMissing configmaps: kube-apiserver-audit-policies-10,kubelet-serving-ca-10,sa-token-signing-certs-10, secrets: etcd-client-10,localhost-recovery-client-token-10,localhost-recovery-serving-certkey-10 From: 18:31:37Z To: 18:31:38Z result=reject event happened 25 times, something is wrong: namespace/openshift-kube-apiserver-operator deployment/kube-apiserver-operator hmsg/9d5ca9c622 - reason/OperatorVersionChanged clusteroperator/kube-apiserver version "kube-apiserver" changed from "1.27.16" to "1.28.15" From: 18:33:48Z To: 18:33:49Z result=reject | |||
| periodic-ci-openshift-release-main-nightly-5.1-e2e-telco5g-cnftests (all) - 2 runs, 100% failed, 50% of failures match = 50% impact | |||
| #2115126726743822336 | junit | 4 hours ago | |
2026-10-27 17:30:29 +0000 UTC: FeatureGatesInitialized etcd-operator FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AWSClusterHostedDNSInstall", "AWSDualStackInstall", "AdditionalStorageConfig", "AuthenticationComponentProxy", "BuildCSIVolumes", "CRIOCredentialProviderConfig", "DualReplica", "EVPN", "EtcdBackendQuota", "EventTTL", "ExternalOIDC", "ExternalOIDCWithUIDAndExtraClaimMappings", "ExternalOIDCWithUpstreamParity", "GCPSovereignCloudInstall", "GatewayAPIWithoutOLM", "ImageStreamImportMode", "IngressControllerDynamicConfigurationManager", "IngressControllerMultipleHAProxyVersions", "InsightsConfig", "InsightsOnDemandDataGather", "IrreconcilableMachineConfig", "KMSEncryption", "KMSv1", "ManagedBootImagesCPMS", "MetricsCollectionProfiles", "MutableCSINodeAllocatableCount", "MutatingAdmissionPolicy", "NewOLM", "NewOLMWebhookProviderOpenshiftServiceCA", "NoRegistryClusterInstall", "OLMLifecycleAndCompatibility", "OSStreams", "SELinuxMount", "SELinuxMountGAReadiness", "ServiceAccountTokenNodeBinding", "StoragePerformantSecurityPolicy", "TLSAdherence", "UpgradeStatus", "VSphereHostVMGroupZonal", "VSphereMixedNodeEnv", "VSphereMultiVCenterDay2"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AWSDedicatedHosts", "AWSEuropeanSovereignCloudInstall", "AuthenticationComponentProxyExternalOIDC", "AutomatedEtcdBackup", "AzureDedicatedHosts", "AzureDualStackInstall", "AzureMultiDisk", "BGPBasedVIPManagement", "BootcNodeManagement", "CBORServingAndStorage", "CRDCompatibilityRequirementOperator", "ClientsAllowCBOR", "ClientsPreferCBOR", "ClusterAPIComputeInstall", "ClusterAPIControlPlaneInstall", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterAPIMachineManagement", "ClusterAPIMachineManagementAWS", "ClusterAPIMachineManagementAzure", "ClusterAPIMachineManagementBareMetal", "ClusterAPIMachineManagementGCP", "ClusterAPIMachineManagementOpenStack", "ClusterAPIMachineManagementPowerVS", "ClusterAPIMachineManagementVSphere", "ClusterAPIOperator", "ClusterMonitoringConfig", "ClusterUpdateAcceptRisks", "ClusterUpdatePreflight", "ClusterVersionOperatorConfiguration", "CompositePodGroup", "ConfidentialCluster", "ConfigurablePKI", "ControllerManagerConfig", "DNSNameResolver", "DRADeviceTaintRules", "DRAWorkloadResourceClaims", "DyanmicServiceEndpointIBMCloud", "EgressIPNodeSelector", "EventedPLEG", "Example", "Example2", "ExternalOIDCAsWebhook", "ExternalOIDCExternalClaimsSourcing", "ExternalSnapshotMetadata", "GCPCustomAPIEndpoints", "GCPCustomAPIEndpointsInstall", "GCPDualStackInstall", "GatewayAPIManagementMode", "GenericWorkload", "GomaxprocsInjection", "HyperShiftOnlyDynamicResourceAllocation", "ImageModeStatusReporting", "IngressComponentRouteLabels", "IngressControllerLBSecurityGroupsAWS", "KarpenterOperator", "MACSecurity", "MachineAPIMigration", "MachineAPIMigrationAWS", "MachineAPIMigrationAzure", "MachineAPIMigrationBareMetal", "MachineAPIMigrationGCP", "MachineAPIMigrationOpenStack", "MachineAPIMigrationPowerVS", "MachineAPIMigrationVSphere", "MachineAPIOperatorDisableMachineHealthCheckController", "ManagedBootImagesAWSCAPI", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "MultiDiskSetup", "MutableTopology", "NetworkConnect", "NetworkObservabilityInstall", "NewOLMBoxCutterRuntime", "NewOLMCatalogdAPIV1Metas", "NewOLMConfigAPI", "NewOLMOwnSingleNamespace", "NoOverlayMode", "NutanixMultiSubnets", "OVNKubernetesUplinkMode", "OVNObservability", "OnPremDNSRecords", "OpenShiftPodSecurityAdmission", "PodGroupPreemptionPolicy", "ProvisioningRequestAvailable", "ShortCertRotation", "SignatureStores", "StoragePathAlert", "TLSGroupPreferences", "TopologyAwareWorkloadScheduling", "UnifiedClusterManagedDNSAndLB", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSpherePerComponentScopedCreds"}}
2026-10-27 17:30:35 +0000 UTC: RequiredInstallerResourcesMissing etcd-operator configmaps: etcd-all-bundles-0,etcd-endpoints-0,etcd-pod-0, secrets: etcd-all-certs-0
2026-10-27 17:30:29 +0000 UTC: MasterNodeObserved etcd-operator Observed new master node cnfdu3-master-1 (UID: 46c769e5-03d0-4755-8c5a-28431fca89f5)
| |||
| #2115126726743822336 | junit | 4 hours ago | |
2026-10-27 17:30:19 +0000 UTC: SignerUpdateRequired kube-apiserver-operator "localhost-recovery-serving-signer" in "openshift-kube-apiserver-operator" requires a new signing cert/key pair: secret doesn't exist 2026-10-27 17:31:28 +0000 UTC: RequiredInstallerResourcesMissing kube-apiserver-operator configmaps: bound-sa-token-signing-certs-0,config-0,etcd-serving-ca-0,kube-apiserver-audit-policies-0,kube-apiserver-cert-syncer-kubeconfig-0,kube-apiserver-pod-0,kubelet-serving-ca-0,sa-token-signing-certs-0, secrets: etcd-client-0,localhost-recovery-client-token-0,localhost-recovery-serving-certkey-0 2026-10-27 17:30:19 +0000 UTC: ConfigMapCreated kube-apiserver-operator Created ConfigMap/sa-token-signing-certs -n openshift-kube-apiserver because it was missing | |||
| #2115126726743822336 | junit | 4 hours ago | |
2026-10-27 17:32:44 +0000 UTC: ConfigMissing kube-apiserver-operator apiServerArguments.etcd-servers has less than two live etcd endpoints: []
2026-10-27 17:35:05 +0000 UTC: RequiredInstallerResourcesMissing kube-apiserver-operator configmaps: bound-sa-token-signing-certs-0,config-0,etcd-serving-ca-0,kube-apiserver-audit-policies-0,kube-apiserver-cert-syncer-kubeconfig-0,kube-apiserver-pod-0,kubelet-serving-ca-0,sa-token-signing-certs-0, secrets: etcd-client-0,localhost-recovery-client-token-0,localhost-recovery-serving-certkey-0
2026-10-27 17:31:51 +0000 UTC: OperatorStatusChanged kube-apiserver-operator Status for clusteroperator/kube-apiserver changed: EvaluationConditionsDetected changed from True to False ("All is well")
| |||
| #2115126726743822336 | junit | 4 hours ago | |
2026-10-27 17:30:23 +0000 UTC: OperatorVersionChanged kube-controller-manager-operator clusteroperator/kube-controller-manager version "raw-internal" changed from "" to "5.1.0-0.nightly-2026-10-26-164535" 2026-10-27 17:31:02 +0000 UTC: RequiredInstallerResourcesMissing kube-controller-manager-operator configmaps: cluster-policy-controller-config-0,config-0,controller-manager-kubeconfig-0,kube-controller-cert-syncer-kubeconfig-0,kube-controller-manager-pod-0,recycler-config-0,service-ca-0,serviceaccount-ca-0, secrets: localhost-recovery-client-token-0,service-account-private-key-0 2026-10-27 17:30:23 +0000 UTC: ObserveFeatureFlagsUpdated kube-controller-manager-operator Updated featureGates to AWSClusterHostedDNSInstall=true,AWSDualStackInstall=true,AdditionalStorageConfig=true,AuthenticationComponentProxy=true,BuildCSIVolumes=true,CRIOCredentialProviderConfig=true,DualReplica=true,EVPN=true,EtcdBackendQuota=true,EventTTL=true,ExternalOIDC=true,ExternalOIDCWithUIDAndExtraClaimMappings=true,ExternalOIDCWithUpstreamParity=true,GCPSovereignCloudInstall=true,GatewayAPIWithoutOLM=true,ImageStreamImportMode=true,IngressControllerDynamicConfigurationManager=true,IngressControllerMultipleHAProxyVersions=true,InsightsConfig=true,InsightsOnDemandDataGather=true,IrreconcilableMachineConfig=true,KMSEncryption=true,KMSv1=true,ManagedBootImagesCPMS=true,MetricsCollectionProfiles=true,MutableCSINodeAllocatableCount=true,MutatingAdmissionPolicy=true,NewOLM=true,NewOLMWebhookProviderOpenshiftServiceCA=true,NoRegistryClusterInstall=true,OLMLifecycleAndCompatibility=true,OSStreams=true,SELinuxMount=true,SELinuxMountGAReadiness=true,ServiceAccountTokenNodeBinding=true,StoragePerformantSecurityPolicy=true,TLSAdherence=true,UpgradeStatus=true,VSphereHostVMGroupZonal=true,VSphereMixedNodeEnv=true,VSphereMultiVCenterDay2=true,AWSClusterHostedDNS=false,AWSDedicatedHosts=false,AWSEuropeanSovereignCloudInstall=false,AuthenticationComponentProxyExternalOIDC=false,AutomatedEtcdBackup=false,AzureDedicatedHosts=false,AzureDualStackInstall=false,AzureMultiDisk=false,BGPBasedVIPManagement=false,BootcNodeManagement=false,CBORServingAndStorage=false,CRDCompatibilityRequirementOperator=false,ClientsAllowCBOR=false,ClientsPreferCBOR=false,ClusterAPIComputeInstall=false,ClusterAPIControlPlaneInstall=false,ClusterAPIInstall=false,ClusterAPIInstallIBMCloud=false,ClusterAPIMachineManagement=false,ClusterAPIMachineManagementAWS=false,ClusterAPIMachineManagementAzure=false,ClusterAPIMachineManagementBareMetal=false,ClusterAPIMachineManagementGCP=false,ClusterAPIMachineManagementOpenStack=false,ClusterAPIMachineManagementPowerVS=false,ClusterAPIMachineManagementVSphere=false,ClusterAPIOperator=false,ClusterMonitoringConfig=false,ClusterUpdateAcceptRisks=false,ClusterUpdatePreflight=false,ClusterVersionOperatorConfiguration=false,CompositePodGroup=false,ConfidentialCluster=false,ConfigurablePKI=false,ControllerManagerConfig=false,DNSNameResolver=false,DRADeviceTaintRules=false,DRAWorkloadResourceClaims=false,DyanmicServiceEndpointIBMCloud=false,EgressIPNodeSelector=false,EventedPLEG=false,Example=false,Example2=false,ExternalOIDCAsWebhook=false,ExternalOIDCExternalClaimsSourcing=false,ExternalSnapshotMetadata=false,GCPCustomAPIEndpoints=false,GCPCustomAPIEndpointsInstall=false,GCPDualStackInstall=false,GatewayAPIManagementMode=false,GenericWorkload=false,GomaxprocsInjection=false,HyperShiftOnlyDynamicResourceAllocation=false,ImageModeStatusReporting=false,IngressComponentRouteLabels=false,IngressControllerLBSecurityGroupsAWS=false,KarpenterOperator=false,MACSecurity=false,MachineAPIMigration=false,MachineAPIMigrationAWS=false,MachineAPIMigrationAzure=false,MachineAPIMigrationBareMetal=false,MachineAPIMigrationGCP=false,MachineAPIMigrationOpenStack=false,MachineAPIMigrationPowerVS=false,MachineAPIMigrationVSphere=false,MachineAPIOperatorDisableMachineHealthCheckController=false,ManagedBootImagesAWSCAPI=false,MaxUnavailableStatefulSet=false,MinimumKubeletVersion=false,MixedCPUsAllocation=false,MultiArchInstallAzure=false,MultiDiskSetup=false,MutableTopology=false,NetworkConnect=false,NetworkObservabilityInstall=false,NewOLMBoxCutterRuntime=false,NewOLMCatalogdAPIV1Metas=false,NewOLMConfigAPI=false,NewOLMOwnSingleNamespace=false,NoOverlayMode=false,NutanixMultiSubnets=false,OVNKubernetesUplinkMode=false,OVNObservability=false,OnPremDNSRecords=false,OpenShiftPodSecurityAdmission=false,PodGroupPreemptionPolicy=false,ProvisioningRequestAvailable=false,ShortCertRotation=false,SignatureStores=false,StoragePathAlert=false,TLSGroupPreferences=false,TopologyAwareWorkloadScheduling=false,UnifiedClusterManagedDNSAndLB=false,VSphereConfigurableMaxAllowedBlockVolumesPerNode=false,VSpherePerComponentScopedCreds=false | |||
| #2115126726743822336 | junit | 4 hours ago | |
2026-10-27 17:30:26 +0000 UTC: MasterNodeObserved openshift-kube-scheduler-operator Observed new master node cnfdu3-master-2 (UID: 4764f7fb-1a73-4d65-abd7-849fbee03b7a)
2026-10-27 17:30:37 +0000 UTC: RequiredInstallerResourcesMissing openshift-kube-scheduler-operator configmaps: config-0,kube-scheduler-cert-syncer-kubeconfig-0,kube-scheduler-pod-0,scheduler-kubeconfig-0,serviceaccount-ca-0, secrets: localhost-recovery-client-token-0
2026-10-27 17:30:26 +0000 UTC: OperatorStatusChanged openshift-kube-scheduler-operator Status for clusteroperator/kube-scheduler changed: Degraded set to Unknown (""),Progressing set to Unknown (""),Available set to Unknown (""),Upgradeable set to Unknown (""),EvaluationConditionsDetected set to Unknown (""),status.relatedObjects changed from [{"operator.openshift.io" "kubeschedulers" "" "cluster"} {"config.openshift.io" "schedulers" "" ""} {"" "namespaces" "" "openshift-config"} {"" "namespaces" "" "openshift-config-managed"} {"" "namespaces" "" "openshift-kube-scheduler-operator"} {"" "namespaces" "" "openshift-kube-scheduler"} {"controlplane.operator.openshift.io" "podnetworkconnectivitychecks" "openshift-kube-scheduler" ""} {"rbac.authorization.k8s.io" "clusterrolebindings" "" "system:openshift:operator:cluster-kube-scheduler-operator"}] to [{"operator.openshift.io" "kubeschedulers" "" "cluster"} {"config.openshift.io" "schedulers" "" ""} {"" "namespaces" "" "openshift-config"} {"" "namespaces" "" "openshift-config-managed"} {"" "namespaces" "" "openshift-kube-scheduler"} {"" "namespaces" "" "openshift-kube-scheduler-operator"} {"controlplane.operator.openshift.io" "podnetworkconnectivitychecks" "openshift-kube-apiserver" ""} {"rbac.authorization.k8s.io" "clusterrolebindings" "" "system:openshift:operator:cluster-kube-scheduler-operator"}],status.versions changed from [] to [{"raw-internal" "5.1.0-0.nightly-2026-10-26-164535"}]
| |||
| #2115126726743822336 | junit | 4 hours ago | |
2026-10-27 17:30:29 +0000 UTC: FeatureGatesInitialized etcd-operator FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AWSClusterHostedDNSInstall", "AWSDualStackInstall", "AdditionalStorageConfig", "AuthenticationComponentProxy", "BuildCSIVolumes", "CRIOCredentialProviderConfig", "DualReplica", "EVPN", "EtcdBackendQuota", "EventTTL", "ExternalOIDC", "ExternalOIDCWithUIDAndExtraClaimMappings", "ExternalOIDCWithUpstreamParity", "GCPSovereignCloudInstall", "GatewayAPIWithoutOLM", "ImageStreamImportMode", "IngressControllerDynamicConfigurationManager", "IngressControllerMultipleHAProxyVersions", "InsightsConfig", "InsightsOnDemandDataGather", "IrreconcilableMachineConfig", "KMSEncryption", "KMSv1", "ManagedBootImagesCPMS", "MetricsCollectionProfiles", "MutableCSINodeAllocatableCount", "MutatingAdmissionPolicy", "NewOLM", "NewOLMWebhookProviderOpenshiftServiceCA", "NoRegistryClusterInstall", "OLMLifecycleAndCompatibility", "OSStreams", "SELinuxMount", "SELinuxMountGAReadiness", "ServiceAccountTokenNodeBinding", "StoragePerformantSecurityPolicy", "TLSAdherence", "UpgradeStatus", "VSphereHostVMGroupZonal", "VSphereMixedNodeEnv", "VSphereMultiVCenterDay2"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AWSDedicatedHosts", "AWSEuropeanSovereignCloudInstall", "AuthenticationComponentProxyExternalOIDC", "AutomatedEtcdBackup", "AzureDedicatedHosts", "AzureDualStackInstall", "AzureMultiDisk", "BGPBasedVIPManagement", "BootcNodeManagement", "CBORServingAndStorage", "CRDCompatibilityRequirementOperator", "ClientsAllowCBOR", "ClientsPreferCBOR", "ClusterAPIComputeInstall", "ClusterAPIControlPlaneInstall", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterAPIMachineManagement", "ClusterAPIMachineManagementAWS", "ClusterAPIMachineManagementAzure", "ClusterAPIMachineManagementBareMetal", "ClusterAPIMachineManagementGCP", "ClusterAPIMachineManagementOpenStack", "ClusterAPIMachineManagementPowerVS", "ClusterAPIMachineManagementVSphere", "ClusterAPIOperator", "ClusterMonitoringConfig", "ClusterUpdateAcceptRisks", "ClusterUpdatePreflight", "ClusterVersionOperatorConfiguration", "CompositePodGroup", "ConfidentialCluster", "ConfigurablePKI", "ControllerManagerConfig", "DNSNameResolver", "DRADeviceTaintRules", "DRAWorkloadResourceClaims", "DyanmicServiceEndpointIBMCloud", "EgressIPNodeSelector", "EventedPLEG", "Example", "Example2", "ExternalOIDCAsWebhook", "ExternalOIDCExternalClaimsSourcing", "ExternalSnapshotMetadata", "GCPCustomAPIEndpoints", "GCPCustomAPIEndpointsInstall", "GCPDualStackInstall", "GatewayAPIManagementMode", "GenericWorkload", "GomaxprocsInjection", "HyperShiftOnlyDynamicResourceAllocation", "ImageModeStatusReporting", "IngressComponentRouteLabels", "IngressControllerLBSecurityGroupsAWS", "KarpenterOperator", "MACSecurity", "MachineAPIMigration", "MachineAPIMigrationAWS", "MachineAPIMigrationAzure", "MachineAPIMigrationBareMetal", "MachineAPIMigrationGCP", "MachineAPIMigrationOpenStack", "MachineAPIMigrationPowerVS", "MachineAPIMigrationVSphere", "MachineAPIOperatorDisableMachineHealthCheckController", "ManagedBootImagesAWSCAPI", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "MultiDiskSetup", "MutableTopology", "NetworkConnect", "NetworkObservabilityInstall", "NewOLMBoxCutterRuntime", "NewOLMCatalogdAPIV1Metas", "NewOLMConfigAPI", "NewOLMOwnSingleNamespace", "NoOverlayMode", "NutanixMultiSubnets", "OVNKubernetesUplinkMode", "OVNObservability", "OnPremDNSRecords", "OpenShiftPodSecurityAdmission", "PodGroupPreemptionPolicy", "ProvisioningRequestAvailable", "ShortCertRotation", "SignatureStores", "StoragePathAlert", "TLSGroupPreferences", "TopologyAwareWorkloadScheduling", "UnifiedClusterManagedDNSAndLB", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSpherePerComponentScopedCreds"}}
2026-10-27 17:30:35 +0000 UTC: RequiredInstallerResourcesMissing etcd-operator configmaps: etcd-all-bundles-0,etcd-endpoints-0,etcd-pod-0, secrets: etcd-all-certs-0
2026-10-27 17:30:29 +0000 UTC: MasterNodeObserved etcd-operator Observed new master node cnfdu3-master-1 (UID: 46c769e5-03d0-4755-8c5a-28431fca89f5)
| |||
| periodic-ci-openshift-release-main-nightly-5.1-e2e-telco5g-sno-cnftests (all) - 2 runs, 100% failed, 100% of failures match = 100% impact | |||
| #2115147412052905984 | junit | 5 hours ago | |
2026-10-27 18:58:25 +0000 UTC: MasterNodeObserved etcd-operator Observed new master node cnfdu5.upstreamci.bos2.lab (UID: 289d1aff-2d58-4dc8-9fff-ee3fe4351b08) 2026-10-27 18:58:34 +0000 UTC: RequiredInstallerResourcesMissing etcd-operator configmaps: etcd-all-bundles-0,etcd-endpoints-0,etcd-pod-0, secrets: etcd-all-certs-0 2026-10-27 18:58:24 +0000 UTC: ServiceAccountCreated etcd-operator Created ServiceAccount/etcd-sa -n openshift-etcd because it was missing | |||
| #2115147412052905984 | junit | 5 hours ago | |
2026-10-27 18:58:10 +0000 UTC: OperatorVersionChanged kube-apiserver-operator clusteroperator/kube-apiserver version "raw-internal" changed from "" to "5.1.0-0.nightly-2026-10-26-164535" 2026-10-27 18:59:02 +0000 UTC: RequiredInstallerResourcesMissing kube-apiserver-operator configmaps: bound-sa-token-signing-certs-0,config-0,etcd-serving-ca-0,kube-apiserver-audit-policies-0,kube-apiserver-cert-syncer-kubeconfig-0,kube-apiserver-pod-0,kubelet-serving-ca-0,sa-token-signing-certs-0, secrets: etcd-client-0,localhost-recovery-client-token-0,localhost-recovery-serving-certkey-0 2026-10-27 18:58:10 +0000 UTC: ConfigMapCreated kube-apiserver-operator Created ConfigMap/bound-sa-token-signing-certs -n openshift-kube-apiserver because it was missing | |||
| #2115147412052905984 | junit | 5 hours ago | |
2026-10-27 18:58:16 +0000 UTC: OperatorVersionChanged kube-controller-manager-operator clusteroperator/kube-controller-manager version "raw-internal" changed from "" to "5.1.0-0.nightly-2026-10-26-164535" 2026-10-27 18:58:42 +0000 UTC: RequiredInstallerResourcesMissing kube-controller-manager-operator configmaps: cluster-policy-controller-config-0,config-0,controller-manager-kubeconfig-0,kube-controller-cert-syncer-kubeconfig-0,kube-controller-manager-pod-0,recycler-config-0,service-ca-0,serviceaccount-ca-0, secrets: localhost-recovery-client-token-0,service-account-private-key-0 2026-10-27 18:58:19 +0000 UTC: MasterNodeObserved kube-controller-manager-operator Observed new master node cnfdu5.upstreamci.bos2.lab (UID: 289d1aff-2d58-4dc8-9fff-ee3fe4351b08) | |||
| #2115147412052905984 | junit | 5 hours ago | |
2026-10-27 19:18:37 +0000 UTC: RevisionTriggered openshift-kube-controller-manager-operator new revision 4 triggered by "required secret/localhost-recovery-client-token has changed"
2026-10-27 19:18:42 +0000 UTC: RequiredInstallerResourcesMissing openshift-kube-controller-manager-operator configmaps: cluster-policy-controller-config-4,config-4,controller-manager-kubeconfig-4,kube-controller-cert-syncer-kubeconfig-4,kube-controller-manager-pod-4,recycler-config-4,service-ca-4,serviceaccount-ca-4
2026-10-27 19:18:40 +0000 UTC: OperatorStatusChanged openshift-kube-controller-manager-operator Status for clusteroperator/kube-controller-manager changed: Degraded message changed from "NodeInstallerDegraded: 1 nodes are failing on revision 3:\nNodeInstallerDegraded: installer: /manifests\",\nNodeInstallerDegraded: Timeout: (time.Duration) 2m0s,\nNodeInstallerDegraded: StaticPodManifestsLockFile: (string) \"\",\nNodeInstallerDegraded: PodMutationFns: ([]installerpod.PodMutationFunc) <nil>,\nNodeInstallerDegraded: KubeletVersion: (string) \"\"\nNodeInstallerDegraded: })\nNodeInstallerDegraded: I1027 19:09:40.907561 1 envvar.go:195] \"Feature gate default state\" feature=\"InOrderInformersBatchProcess\" enabled=true\nNodeInstallerDegraded: I1027 19:09:40.907590 1 envvar.go:195] \"Feature gate default state\" feature=\"UnlockWhileProcessingFIFO\" enabled=true\nNodeInstallerDegraded: I1027 19:09:40.907598 1 envvar.go:195] \"Feature gate default state\" feature=\"InformerResourceVersion\" enabled=true\nNodeInstallerDegraded: I1027 19:09:40.907605 1 envvar.go:195] \"Feature gate default state\" feature=\"AtomicFIFO\" enabled=true\nNodeInstallerDegraded: I1027 19:09:40.907611 1 envvar.go:195] \"Feature gate default state\" feature=\"ClientsAllowCBOR\" enabled=false\nNodeInstallerDegraded: I1027 19:09:40.907618 1 envvar.go:195] \"Feature gate default state\" feature=\"WatchListClient\" enabled=true\nNodeInstallerDegraded: I1027 19:09:40.907624 1 envvar.go:195] \"Feature gate default state\" feature=\"ClientsPreferCBOR\" enabled=false\nNodeInstallerDegraded: I1027 19:09:40.907629 1 envvar.go:195] \"Feature gate default state\" feature=\"ClientsAllowCARotation\" enabled=true\nNodeInstallerDegraded: I1027 19:09:40.907634 1 envvar.go:195] \"Feature gate default state\" feature=\"ClientsAllowTLSCacheGC\" enabled=true\nNodeInstallerDegraded: I1027 19:09:40.907639 1 envvar.go:195] \"Feature gate default state\" feature=\"InOrderInformers\" enabled=true\nNodeInstallerDegraded: I1027 19:09:40.909262 1 cmd.go:428] Getting controller reference for node cnfdu5.upstreamci.bos2.lab\nNodeInstallerDegraded: I1027 19:09:41.098240 1 cmd.go:441] Waiting for installer revisions to settle for node cnfdu5.upstreamci.bos2.lab\nNodeInstallerDegraded: I1027 19:09:41.102335 1 cmd.go:533] Waiting additional period after revisions have settled for node cnfdu5.upstreamci.bos2.lab\nNodeInstallerDegraded: I1027 19:10:11.102391 1 cmd.go:539] Getting installer pods for node cnfdu5.upstreamci.bos2.lab\nNodeInstallerDegraded: F1027 19:10:25.106982 1 cmd.go:113] Get \"https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/pods?labelSelector=app%3Dinstaller\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)\nNodeInstallerDegraded: \nNodeControllerDegraded: All master nodes are ready" to "RevisionControllerDegraded: unable to ApplyStatus for operator using fieldManager \"kube-controller-manager-RevisionController\": KubeControllerManager.operator.openshift.io \"cluster\" is invalid: status.latestAvailableRevision: Invalid value: 3: must only increase\nInstallerControllerDegraded: missing required resources: configmaps: cluster-policy-controller-config-4,config-4,controller-manager-kubeconfig-4,kube-controller-cert-syncer-kubeconfig-4,kube-controller-manager-pod-4,recycler-config-4,service-ca-4,serviceaccount-ca-4\nNodeControllerDegraded: All master nodes are ready",Progressing message changed from "NodeInstallerProgressing: 1 node is at revision 0; 0 nodes have achieved new revision 3" to "NodeInstallerProgressing: 1 node is at revision 3; 0 nodes have achieved new revision 4",Available changed from False to True ("StaticPodsAvailable: 1 nodes are active; 1 node is at revision 3; 0 nodes have achieved new revision 4")
| |||
| #2115147412052905984 | junit | 5 hours ago | |
2026-10-27 18:58:17 +0000 UTC: MasterNodeObserved openshift-kube-scheduler-operator Observed new master node cnfdu5.upstreamci.bos2.lab (UID: 289d1aff-2d58-4dc8-9fff-ee3fe4351b08) 2026-10-27 18:58:29 +0000 UTC: RequiredInstallerResourcesMissing openshift-kube-scheduler-operator configmaps: config-0,kube-scheduler-cert-syncer-kubeconfig-0,kube-scheduler-pod-0,scheduler-kubeconfig-0,serviceaccount-ca-0, secrets: localhost-recovery-client-token-0 2026-10-27 18:58:17 +0000 UTC: ConfigMapCreated openshift-kube-scheduler-operator Created ConfigMap/config -n openshift-kube-scheduler because it was missing | |||
| #2115147412052905984 | junit | 5 hours ago | |
2026-10-27 18:58:25 +0000 UTC: MasterNodeObserved etcd-operator Observed new master node cnfdu5.upstreamci.bos2.lab (UID: 289d1aff-2d58-4dc8-9fff-ee3fe4351b08) 2026-10-27 18:58:34 +0000 UTC: RequiredInstallerResourcesMissing etcd-operator configmaps: etcd-all-bundles-0,etcd-endpoints-0,etcd-pod-0, secrets: etcd-all-certs-0 2026-10-27 18:58:24 +0000 UTC: ServiceAccountCreated etcd-operator Created ServiceAccount/etcd-sa -n openshift-etcd because it was missing | |||
| #2114779260467023872 | junit | 29 hours ago | |
2026-10-26 18:35:22 +0000 UTC: OperatorVersionChanged etcd-operator clusteroperator/etcd version "raw-internal" changed from "" to "5.1.0-0.nightly-2026-10-21-040310" 2026-10-26 18:35:32 +0000 UTC: RequiredInstallerResourcesMissing etcd-operator configmaps: etcd-all-bundles-0,etcd-endpoints-0,etcd-pod-0, secrets: etcd-all-certs-0 2026-10-26 18:35:22 +0000 UTC: ObservedConfigChanged etcd-operator Writing updated observed config: @@ -1 +1,19 @@ | |||
| #2114779260467023872 | junit | 29 hours ago | |
2026-10-26 18:35:14 +0000 UTC: OperatorVersionChanged kube-apiserver-operator clusteroperator/kube-apiserver version "raw-internal" changed from "" to "5.1.0-0.nightly-2026-10-21-040310" 2026-10-26 18:35:57 +0000 UTC: RequiredInstallerResourcesMissing kube-apiserver-operator configmaps: bound-sa-token-signing-certs-0,config-0,etcd-serving-ca-0,kube-apiserver-audit-policies-0,kube-apiserver-cert-syncer-kubeconfig-0,kube-apiserver-pod-0,kubelet-serving-ca-0,sa-token-signing-certs-0, secrets: etcd-client-0,localhost-recovery-client-token-0,localhost-recovery-serving-certkey-0 2026-10-26 18:35:14 +0000 UTC: SecretUpdated kube-apiserver-operator Updated Secret/service-network-serving-signer -n openshift-kube-apiserver-operator because it changed | |||
| #2114779260467023872 | junit | 29 hours ago | |
2026-10-26 18:35:17 +0000 UTC: ServiceAccountCreated kube-controller-manager-operator Created ServiceAccount/installer-sa -n openshift-kube-controller-manager because it was missing 2026-10-26 18:35:45 +0000 UTC: RequiredInstallerResourcesMissing kube-controller-manager-operator configmaps: cluster-policy-controller-config-0,config-0,controller-manager-kubeconfig-0,kube-controller-cert-syncer-kubeconfig-0,kube-controller-manager-pod-0,recycler-config-0,service-ca-0,serviceaccount-ca-0, secrets: localhost-recovery-client-token-0,service-account-private-key-0 2026-10-26 18:35:17 +0000 UTC: OperatorVersionChanged kube-controller-manager-operator clusteroperator/kube-controller-manager version "raw-internal" changed from "" to "5.1.0-0.nightly-2026-10-21-040310" | |||
| #2114779260467023872 | junit | 29 hours ago | |
2026-10-26 18:35:26 +0000 UTC: MasterNodeObserved openshift-kube-scheduler-operator Observed new master node cnfdu6.upstreamci.bos2.lab (UID: aec01108-b9f3-41aa-a356-d4f0844d4b89) 2026-10-26 18:35:37 +0000 UTC: RequiredInstallerResourcesMissing openshift-kube-scheduler-operator configmaps: config-0,kube-scheduler-cert-syncer-kubeconfig-0,kube-scheduler-pod-0,scheduler-kubeconfig-0,serviceaccount-ca-0, secrets: localhost-recovery-client-token-0 2026-10-26 18:35:26 +0000 UTC: ConfigMapCreated openshift-kube-scheduler-operator Created ConfigMap/config -n openshift-kube-scheduler because it was missing | |||
| #2114779260467023872 | junit | 29 hours ago | |
2026-10-26 18:35:22 +0000 UTC: OperatorVersionChanged etcd-operator clusteroperator/etcd version "raw-internal" changed from "" to "5.1.0-0.nightly-2026-10-21-040310" 2026-10-26 18:35:32 +0000 UTC: RequiredInstallerResourcesMissing etcd-operator configmaps: etcd-all-bundles-0,etcd-endpoints-0,etcd-pod-0, secrets: etcd-all-certs-0 2026-10-26 18:35:22 +0000 UTC: ObservedConfigChanged etcd-operator Writing updated observed config: @@ -1 +1,19 @@ | |||
| #2114779260467023872 | junit | 29 hours ago | |
2026-10-26 18:35:14 +0000 UTC: OperatorVersionChanged kube-apiserver-operator clusteroperator/kube-apiserver version "raw-internal" changed from "" to "5.1.0-0.nightly-2026-10-21-040310" 2026-10-26 18:35:57 +0000 UTC: RequiredInstallerResourcesMissing kube-apiserver-operator configmaps: bound-sa-token-signing-certs-0,config-0,etcd-serving-ca-0,kube-apiserver-audit-policies-0,kube-apiserver-cert-syncer-kubeconfig-0,kube-apiserver-pod-0,kubelet-serving-ca-0,sa-token-signing-certs-0, secrets: etcd-client-0,localhost-recovery-client-token-0,localhost-recovery-serving-certkey-0 2026-10-26 18:35:14 +0000 UTC: SecretUpdated kube-apiserver-operator Updated Secret/service-network-serving-signer -n openshift-kube-apiserver-operator because it changed | |||
| periodic-ci-openshift-release-main-ci-4.15-e2e-gcp-sdn-upgrade-out-of-change (all) - 1 runs, 0% failed, 100% of runs match | |||
| #2115128105080197120 | junit | 5 hours ago | |
# [bz-etcd] pathological event should not see excessive RequiredInstallerResourcesMissing secrets event [namespace/openshift-etcd-operator deployment/etcd-operator hmsg/7759db15ed - configmaps: etcd-metrics-proxy-client-ca,etcd-metrics-proxy-serving-ca,etcd-peer-client-ca,etcd-scripts,etcd-serving-ca,restore-etcd-pod, secrets: etcd-all-certs, configmaps: etcd-endpoints-8,etcd-metrics-proxy-client-ca-8,etcd-metrics-proxy-serving-ca-8,etcd-peer-client-ca-8,etcd-pod-8,etcd-serving-ca-8, secrets: etcd-all-certs-8] happened 11 times | |||
| aggregator-periodic-ci-openshift-release-main-ci-5.1-upgrade-from-stable-5.0-e2e-gcp-ovn-rt-upgrade (all) - 2 runs, 50% failed, 100% of failures match = 50% impact | |||
| #2115082906006196224 | junit | 6 hours ago | |
# openshift-tests.[Monitor:legacy-etcd-invariants][bz-etcd] pathological event should not see excessive RequiredInstallerResourcesMissing secrets name: '[Monitor:legacy-etcd-invariants][bz-etcd] pathological event should not see excessive RequiredInstallerResourcesMissing secrets' testsuitename: openshift-tests | |||
| #2115082906006196224 | junit | 6 hours ago | |
# openshift-tests-upgrade.[Monitor:legacy-etcd-invariants][bz-etcd] pathological event should not see excessive RequiredInstallerResourcesMissing secrets name: '[Monitor:legacy-etcd-invariants][bz-etcd] pathological event should not see excessive RequiredInstallerResourcesMissing secrets' testsuitename: openshift-tests-upgrade | |||
| periodic-ci-openshift-release-main-nightly-4.22-e2e-aws-ovn-single-node-serial (all) - 3 runs, 0% failed, 33% of runs match | |||
| #2115102512020197376 | junit | 7 hours ago | |
# [Monitor:legacy-etcd-invariants][bz-etcd] pathological event should not see excessive RequiredInstallerResourcesMissing secrets event [namespace/openshift-kube-apiserver-operator deployment/kube-apiserver-operator hmsg/b6fa46dc1a - secrets: etcd-client-11,localhost-recovery-client-token-11,localhost-recovery-serving-certkey-11] happened 11 times | |||
| periodic-ci-openshift-cluster-authentication-operator-release-4.20-periodics-e2e-aws-sno-external-oidc-configure-techpreview (all) - 2 runs, 0% failed, 50% of runs match | |||
| #2115103156357566464 | junit | 8 hours ago | |
# [bz-etcd] pathological event should not see excessive RequiredInstallerResourcesMissing secrets event [namespace/openshift-kube-apiserver-operator deployment/kube-apiserver-operator hmsg/e876ad080f - secrets: etcd-client-30,localhost-recovery-client-token-30,localhost-recovery-serving-certkey-30] happened 11 times | |||
| pull-ci-openshift-cluster-kube-apiserver-operator-main-e2e-gcp-operator-encryption-rotation-single-node-ote (all) - 2 runs, 100% failed, 50% of failures match = 50% impact | |||
| #2115078962609655808 | junit | 10 hours ago | |
# [Monitor:legacy-etcd-invariants][bz-etcd] pathological event should not see excessive RequiredInstallerResourcesMissing secrets event [namespace/openshift-kube-apiserver-operator deployment/kube-apiserver-operator hmsg/2416ebc3f8 - configmaps: bound-sa-token-signing-certs-10,config-10,etcd-serving-ca-10,kube-apiserver-audit-policies-10,kube-apiserver-cert-syncer-kubeconfig-10,kube-apiserver-pod-10,kubelet-serving-ca-10,sa-token-signing-certs-10] happened 11 times | |||
| periodic-ci-openshift-release-main-ci-4.16-e2e-aws-ovn-upgrade (all) - 5 runs, 20% failed, 100% of failures match = 20% impact | |||
| #2114968905175797760 | junit | 15 hours ago | |
# [bz-etcd] pathological event should not see excessive RequiredInstallerResourcesMissing secrets event [namespace/openshift-etcd-operator deployment/etcd-operator hmsg/42efa8a8de - secrets: etcd-all-certs, secrets: etcd-all-certs-9] happened 11 times | |||
| periodic-ci-openshift-cluster-authentication-operator-release-4.22-periodics-e2e-aws-sno-external-oidc-revertoauth (all) - 2 runs, 0% failed, 50% of runs match | |||
| #2114841018305089536 | junit | 26 hours ago | |
# [Monitor:legacy-etcd-invariants][bz-etcd] pathological event should not see excessive RequiredInstallerResourcesMissing secrets event [namespace/openshift-kube-apiserver-operator deployment/kube-apiserver-operator hmsg/187d0cac4b - secrets: etcd-client-10,localhost-recovery-client-token-10,localhost-recovery-serving-certkey-10] happened 12 times | |||
| periodic-ci-openshift-multiarch-main-nightly-4.15-upgrade-from-stable-4.14-ocp-e2e-upgrade-azure-ovn-arm64 (all) - 2 runs, 50% failed, 100% of failures match = 50% impact | |||
| #2114733972352143360 | junit | 32 hours ago | |
# [bz-etcd] pathological event should not see excessive RequiredInstallerResourcesMissing secrets event [namespace/openshift-etcd-operator deployment/etcd-operator hmsg/344cad4d1c - configmaps: etcd-metrics-proxy-client-ca,etcd-metrics-proxy-serving-ca,etcd-peer-client-ca,etcd-scripts,etcd-serving-ca,restore-etcd-pod, secrets: etcd-all-certs, configmaps: etcd-endpoints-9,etcd-metrics-proxy-client-ca-9,etcd-metrics-proxy-serving-ca-9,etcd-peer-client-ca-9,etcd-pod-9,etcd-serving-ca-9, secrets: etcd-all-certs-9] happened 11 times | |||
| periodic-ci-openshift-release-main-ci-4.17-upgrade-from-stable-4.16-e2e-azure-ovn-upgrade (all) - 4 runs, 50% failed, 50% of failures match = 25% impact | |||
| #2114704345504157696 | junit | 33 hours ago | |
# [bz-etcd] pathological event should not see excessive RequiredInstallerResourcesMissing secrets event [namespace/openshift-kube-apiserver-operator deployment/kube-apiserver-operator hmsg/c7edebf87e - configmaps: kube-apiserver-audit-policies-11, secrets: etcd-client-11,localhost-recovery-client-token-11,localhost-recovery-serving-certkey-11] happened 21 times | |||
| aggregated-gcp-ovn-rt-upgrade-5.0-major-release-openshift-release-analysis-aggregator (all) - 13 runs, 69% failed, 33% of failures match = 23% impact | |||
| #2114738353768239104 | junit | 34 hours ago | |
# openshift-tests.[Monitor:legacy-etcd-invariants][bz-etcd] pathological event should not see excessive RequiredInstallerResourcesMissing secrets name: '[Monitor:legacy-etcd-invariants][bz-etcd] pathological event should not see excessive RequiredInstallerResourcesMissing secrets' testsuitename: openshift-tests | |||
| #2114738353768239104 | junit | 34 hours ago | |
# openshift-tests-upgrade.[Monitor:legacy-etcd-invariants][bz-etcd] pathological event should not see excessive RequiredInstallerResourcesMissing secrets name: '[Monitor:legacy-etcd-invariants][bz-etcd] pathological event should not see excessive RequiredInstallerResourcesMissing secrets' testsuitename: openshift-tests-upgrade | |||
| #2114734951722127360 | junit | 34 hours ago | |
# openshift-tests.[Monitor:legacy-etcd-invariants][bz-etcd] pathological event should not see excessive RequiredInstallerResourcesMissing secrets name: '[Monitor:legacy-etcd-invariants][bz-etcd] pathological event should not see excessive RequiredInstallerResourcesMissing secrets' testsuitename: openshift-tests | |||
| #2114734951722127360 | junit | 34 hours ago | |
# openshift-tests-upgrade.[Monitor:legacy-etcd-invariants][bz-etcd] pathological event should not see excessive RequiredInstallerResourcesMissing secrets name: '[Monitor:legacy-etcd-invariants][bz-etcd] pathological event should not see excessive RequiredInstallerResourcesMissing secrets' testsuitename: openshift-tests-upgrade | |||
| #2114659141883531264 | junit | 35 hours ago | |
# openshift-tests.[Monitor:legacy-etcd-invariants][bz-etcd] pathological event should not see excessive RequiredInstallerResourcesMissing secrets name: '[Monitor:legacy-etcd-invariants][bz-etcd] pathological event should not see excessive RequiredInstallerResourcesMissing secrets' testsuitename: openshift-tests | |||
| #2114659141883531264 | junit | 35 hours ago | |
# openshift-tests-upgrade.[Monitor:legacy-etcd-invariants][bz-etcd] pathological event should not see excessive RequiredInstallerResourcesMissing secrets name: '[Monitor:legacy-etcd-invariants][bz-etcd] pathological event should not see excessive RequiredInstallerResourcesMissing secrets' testsuitename: openshift-tests-upgrade | |||
| pull-ci-openshift-cluster-etcd-operator-release-4.15-e2e-aws-ovn-etcd-scaling (all) - 1 runs, 100% failed, 100% of failures match = 100% impact | |||
| #2114686873946296320 | junit | 36 hours ago | |
# [bz-etcd] pathological event should not see excessive RequiredInstallerResourcesMissing secrets event [namespace/openshift-etcd-operator deployment/etcd-operator hmsg/7462bbc2a0 - configmaps: etcd-endpoints-14,etcd-metrics-proxy-client-ca-14,etcd-metrics-proxy-serving-ca-14,etcd-peer-client-ca-14,etcd-serving-ca-14, secrets: etcd-all-certs-14] happened 21 times | |||
| periodic-ci-openshift-cluster-authentication-operator-release-4.21-periodics-e2e-aws-sno-external-oidc-uid-extra (all) - 2 runs, 0% failed, 50% of runs match | |||
| #2114690021318463488 | junit | 36 hours ago | |
# [Monitor:legacy-etcd-invariants][bz-etcd] pathological event should not see excessive RequiredInstallerResourcesMissing secrets event [namespace/openshift-kube-apiserver-operator deployment/kube-apiserver-operator hmsg/f60a33e686 - configmaps: bound-sa-token-signing-certs-13,config-13,etcd-serving-ca-13,kube-apiserver-audit-policies-13,kube-apiserver-cert-syncer-kubeconfig-13,kube-apiserver-pod-13,kubelet-serving-ca-13,sa-token-signing-certs-13] happened 11 times | |||
| pull-ci-openshift-cluster-control-plane-machine-set-operator-release-4.22-e2e-aws-ovn-etcd-scaling (all) - 1 runs, 100% failed, 100% of failures match = 100% impact | |||
| #2114643137853919232 | junit | 36 hours ago | |
# [Monitor:legacy-etcd-invariants][bz-etcd] pathological event should not see excessive RequiredInstallerResourcesMissing secrets event [namespace/openshift-kube-apiserver-operator deployment/kube-apiserver-operator hmsg/d933c85ad2 - secrets: etcd-client-12,localhost-recovery-client-token-12,localhost-recovery-serving-certkey-12] happened 21 times | |||
| periodic-ci-openshift-cluster-authentication-operator-release-5.0-periodics-e2e-aws-sno-external-oidc-upstream-parity-techpreview (all) - 2 runs, 0% failed, 50% of runs match | |||
| #2114642710143963136 | junit | 38 hours ago | |
# [Monitor:legacy-etcd-invariants][bz-etcd] pathological event should not see excessive RequiredInstallerResourcesMissing secrets event [namespace/openshift-kube-apiserver-operator deployment/kube-apiserver-operator hmsg/87b0c047db - secrets: etcd-client-26,localhost-recovery-client-token-26,localhost-recovery-serving-certkey-26] happened 11 times | |||
| periodic-ci-openshift-release-main-stable-4.y-e2e-aws-ovn-upgrade (all) - 51 runs, 41% failed, 5% of failures match = 2% impact | |||
| #2114585481550237696 | junit | 41 hours ago | |
event happened 31 times, something is wrong: namespace/openshift-kube-controller-manager-operator deployment/kube-controller-manager-operator hmsg/adeb3c9d54 - reason/RequiredInstallerResourcesMissing configmaps: recycler-config-11, secrets: localhost-recovery-client-token-11,service-account-private-key-11 (06:38:14Z) result=reject | |||
| periodic-ci-openshift-release-main-stable-4.y-e2e-azure-ovn-upgrade (all) - 34 runs, 71% failed, 8% of failures match = 6% impact | |||
| #2114585489112567808 | junit | 41 hours ago | |
event happened 51 times, something is wrong: namespace/openshift-kube-apiserver-operator deployment/kube-apiserver-operator hmsg/e26544164e - reason/RequiredInstallerResourcesMissing configmaps: bound-sa-token-signing-certs-9,config-9,etcd-serving-ca-9,kube-apiserver-audit-policies-9,kube-apiserver-cert-syncer-kubeconfig-9,kubelet-serving-ca-9,sa-token-signing-certs-9, secrets: etcd-client-9,localhost-recovery-client-token-9,localhost-recovery-serving-certkey-9 (06:35:52Z) result=reject | |||
| #2114585473157435392 | junit | 41 hours ago | |
# [bz-etcd] pathological event should not see excessive RequiredInstallerResourcesMissing secrets event [namespace/openshift-etcd-operator deployment/etcd-operator hmsg/1f6f58de6d - secrets: etcd-all-certs, secrets: etcd-all-certs-12] happened 11 times | |||
Found in 0.04% of runs (0.26% of failures) across 70471 total runs and 14324 jobs (15.95% failed) in 1.11s - clear search | chart view - source code located on github