<p>You can follow below approach:</p>
<ol>
<li>Use <a href="https://github.com/kubernetes/autoscaler" rel="nofollow noreferrer">Cluster Autoscaler</a> which should scale-in or scale-out nodes as and when required.</li>
<li>Create nodegroups with multiple instance types so that you always get a spot node.</li>
<li>Use labels and anti-affinity rules to schedule applications on any of these available spot nodes.</li>
</ol>

CC BY-SA 4.0
2021-06-07T17:17:29.917Z
Hi Samit, I have already done the same as mentioned in my question. This does not answer my use case.
rohatgisanat
CC BY-SA 4.0
2021-06-03T18:09:20.670Z
2021-06-03T18:09:20.670Z
Samit
<p>i need to deploy a k8s deployment to be stretched across two kinds of nodes, dedicated and spot nodes.</p>
<p>The end-goal is to have deployment which has some pods on dedicated nodes, but when scaled uses spot nodes.</p>
<p><strong>Example</strong>:-</p>
<p>A deployment has 2 pods (min) and is scalable up to 5 pods (max).The deployment should behave such that the first 2 pods are always scheduled on dedicated nodes and the rest 3 pods (5-2) are always deployed on spot nodes.</p>
<p><strong>Use case</strong>:- To improve cost/reliability of my platform.</p>
<ul>
<li><strong>Cost</strong>:- A prefixed set of pods running at idle vs the max pods running
at maximum load gives me an idea of how much my pods are scaling. It also ensures that the end of the day when load subsides,the node counts return back to a fixed predictable count.</li>
<li><strong>Reliability</strong>:- I want to reduce the possibility of outages when the
spot nodes are taken away.Ensuring that atleast 2 pods are
running on a dedicated nodes, give me confidence that my application
does not face an outages should all the spot nodes get deleted.(It does happen!)</li>
</ul>
<p><strong>What I have tried so far:-</strong></p>
<ul>
<li><strong>Labels and Node Affinity</strong>:- Node affinity ascertains where the pods
will be scheduled. But this approach forces me to select <em>either</em>
dedicated <em>or</em> spot nodes. (not a combination of both)</li>
<li><strong>Pod Anti Affinity</strong>:- It works somewhat,it deploy one pod/node till the number of pods = number of dedicated nodes, the starts deployment on spot. But this forces me to limit the size of dedicated nodepool. Also one pod/node is not the use case I am looking for.</li>
</ul>
<p><strong>Understanding of the problem:-</strong></p>
<ul>
<li><p>It seems like a common enough use case where atleast some portion of
pods are always running but I have not seen a solution to this
anywhere.</p>
</li>
<li><p>It may go against the k8s philosophy where the app should be
unaware/not tightly coupled with the infrastructure below but I am
not sure if this is the case here.</p>
</li>
<li><p>Running a free for all nodes (no affinity),can cause unbalanced deployments, where scaled pods of one deployment, may prevent any pods of another deployment from getting dedicated node resources.</p>
</li>
</ul>
<p>Let me know if this is achievable and if not , how i can reduce the risk of app outage by running on all spot nodes.</p>

<kubernetes><architecture>
Scale up pods of k8s deployment across spot and dedicated instances
rohatgisanat
# Scale up pods of k8s deployment across spot and dedicated instances

(Asked by: username_0 on 2021-06-03)

i need to deploy a k8s deployment to be stretched across two kinds of nodes, dedicated and spot nodes.

The end-goal is to have deployment which has some pods on dedicated nodes, but when scaled uses spot nodes.

**Example**:-

A deployment has 2 pods (min) and is scalable up to 5 pods (max).The deployment should behave such that the first 2 pods are always scheduled on dedicated nodes and the rest 3 pods (5-2) are always deployed on spot nodes.

**Use case**:- To improve cost/reliability of my platform.

* **Cost**:- A prefixed set of pods running at idle vs the max pods running
  at maximum load gives me an idea of how much my pods are scaling. It also ensures that the end of the day when load subsides,the node counts return back to a fixed predictable count.
* **Reliability**:- I want to reduce the possibility of outages when the
  spot nodes are taken away.Ensuring that atleast 2 pods are
  running on a dedicated nodes, give me confidence that my application
  does not face an outages should all the spot nodes get deleted.(It does happen!)

**What I have tried so far:-**

* **Labels and Node Affinity**:- Node affinity ascertains where the pods
  will be scheduled. But this approach forces me to select *either*
  dedicated *or* spot nodes. (not a combination of both)
* **Pod Anti Affinity**:- It works somewhat,it deploy one pod/node till the number of pods = number of dedicated nodes, the starts deployment on spot. But this forces me to limit the size of dedicated nodepool. Also one pod/node is not the use case I am looking for.

**Understanding of the problem:-**

* It seems like a common enough use case where atleast some portion of
  pods are always running but I have not seen a solution to this
  anywhere.
* It may go against the k8s philosophy where the app should be
  unaware/not tightly coupled with the infrastructure below but I am
  not sure if this is the case here.
* Running a free for all nodes (no affinity),can cause unbalanced deployments, where scaled pods of one deployment, may prevent any pods of another deployment from getting dedicated node resources.

Let me know if this is achievable and if not , how i can reduce the risk of app outage by running on all spot nodes.

**Tags:** kubernetes, architecture


---

## 1 answers

### Answer by username_1 on 2021-06-03. Score: 1

You can follow below approach:

1. Use [Cluster Autoscaler](https://github.com/kubernetes/autoscaler) which should scale-in or scale-out nodes as and when required.
2. Create nodegroups with multiple instance types so that you always get a spot node.
3. Use labels and anti-affinity rules to schedule applications on any of these available spot nodes.

> **Comments:**
> **username_0**: Hi username_1, I have already done the same as mentioned in my question. This does not answer my use case.  

---

