<p>Seems that the problem was with the <code>replicas: {{ .Values.replicaCount }}</code> definition. It seems that, if you are using HPA, <code>replicas</code> can't be used. I removed this line and the HPA started scaling.</p>

CC BY-SA 4.0
2022-07-04T17:54:05.773Z
2022-07-04T17:54:05.773Z
Alexandru Constantin Arvinte
<p>So I have a <code>HorizontalPodAutoscaler</code> set up for my backend (an fpm-server and an Nginx server for a Laravel application).</p>
<p>The problem is that when the <code>HPA</code> is under load, it scales up the pods but it terminates them instantly, not even letting them get into the <strong>Running</strong> state.</p>
<p>The metrics are good, the scale-up behavior is as expected the only problem is that the pods get terminated right after scaling.</p>
<p>What could be the problem?</p>
<p><strong>Edit:</strong> The same <code>HPA</code> is used on the frontend and it's working as expected, the problem seems to be only on the backend.</p>
<p><strong>Edit 2:</strong> I have <code>Cluster Autoscaler</code> enabled, it does it's job, nodes get added when they are needed and then cleaned up so it's not an issue about available resources.</p>
<p><strong>deployment.yaml</strong></p>
<pre><code>apiVersion: apps/v1
kind: Deployment
metadata:
  name: fpm-server
  labels:
    tier: backend
    layer: fpm
spec:
  replicas: {{ .Values.replicaCount }}
  selector:
    matchLabels:
      tier: backend
      layer: fpm
  template:
    metadata:
      labels:
        tier: backend
        layer: fpm
    spec:
      {{- with .Values.imagePullSecrets }}
      imagePullSecrets:
        {{- toYaml . | nindent 8 }}
      {{- end }}           
      containers:
        - name: fpm
          image: &quot;{{ .Values.fpm.image.repository }}:{{ .Values.fpm.image.tag }}&quot;
          ports:
            - name: http
              containerPort: 9000
              protocol: TCP
          env:
          {{- range $name, $value := .Values.env }}
          - name: {{ $name }}
            value: &quot;{{ $value }}&quot; 
          {{- end }}             
          envFrom:
            - secretRef:
                name: backend-secrets
          resources:
            {{- toYaml .Values.resources | nindent 12 }}
</code></pre>
<p><strong>hpa.yaml</strong></p>
<pre><code>apiVersion: autoscaling/v2beta1
kind: HorizontalPodAutoscaler
metadata:
  name: fpm-server-hpa
  labels:
    tier: backend
    layer: fpm
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: fpm-server
  minReplicas: {{ .Values.autoscaling.minReplicas }}
  maxReplicas: {{ .Values.autoscaling.maxReplicas }}
  metrics:
    {{- if .Values.autoscaling.targetCPUUtilizationPercentage }}
    - type: Resource
      resource:
        name: cpu
        targetAverageUtilization: {{ .Values.autoscaling.targetCPUUtilizationPercentage }}
    {{- end }}
    {{- if .Values.autoscaling.targetMemoryUtilizationPercentage }}
    - type: Resource
      resource:
        name: memory
        targetAverageUtilization: {{ .Values.autoscaling.targetMemoryUtilizationPercentage }}
    {{- end }}
</code></pre>

<kubernetes>
HorizontalPodAutoscaler scales up pods but then terminates them instantly
Alexandru Constantin Arvinte
# HorizontalPodAutoscaler scales up pods but then terminates them instantly

(Asked by: username_0 on 2022-07-02)

So I have a HorizontalPodAutoscaler set up for my backend (an fpm-server and an Nginx server for a Laravel application).
The problem is that when the HPA is under load, it scales up the pods but it terminates them instantly, not even letting them get into the Running state.
The metrics are good, the scale-up behavior is as expected the only problem is that the pods get terminated right after scaling.
What could be the problem?
Edit: The same HPA is used on the frontend and it's working as expected, the problem seems to be only on the backend.
Edit 2: I have Cluster Autoscaler enabled, it does it's job, nodes get added when they are needed and then cleaned up so it's not an issue about available resources.
deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: fpm-server
  labels:
    tier: backend
    layer: fpm
spec:
  replicas: {{ .Values.replicaCount }}
  selector:
    matchLabels:
      tier: backend
      layer: fpm
  template:
    metadata:
      labels:
        tier: backend
        layer: fpm
    spec:
      {{- with .Values.imagePullSecrets }}
      imagePullSecrets:
        {{- toYaml . | nindent 8 }}
      {{- end }}           
      containers:
        - name: fpm
          image: "{{ .Values.fpm.image.repository }}:{{ .Values.fpm.image.tag }}"
          ports:
            - name: http
              containerPort: 9000
              protocol: TCP
          env:
          {{- range $name, $value := .Values.env }}
          - name: {{ $name }}
            value: "{{ $value }}" 
          {{- end }}             
          envFrom:
            - secretRef:
                name: backend-secrets
          resources:
            {{- toYaml .Values.resources | nindent 12 }}

hpa.yaml
apiVersion: autoscaling/v2beta1
kind: HorizontalPodAutoscaler
metadata:
  name: fpm-server-hpa
  labels:
    tier: backend
    layer: fpm
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: fpm-server
  minReplicas: {{ .Values.autoscaling.minReplicas }}
  maxReplicas: {{ .Values.autoscaling.maxReplicas }}
  metrics:
    {{- if .Values.autoscaling.targetCPUUtilizationPercentage }}
    - type: Resource
      resource:
        name: cpu
        targetAverageUtilization: {{ .Values.autoscaling.targetCPUUtilizationPercentage }}
    {{- end }}
    {{- if .Values.autoscaling.targetMemoryUtilizationPercentage }}
    - type: Resource
      resource:
        name: memory
        targetAverageUtilization: {{ .Values.autoscaling.targetMemoryUtilizationPercentage }}
    {{- end }}

**Tags:** kubernetes


---

## 1 answers

### Answer by username_0 on 2022-07-04. Score: 1 | Accepted answer

Seems that the problem was with the `replicas: {{ .Values.replicaCount }}` definition. It seems that, if you are using HPA, `replicas` can't be used. I removed this line and the HPA started scaling.


---

