Q: How scheduledThreadPool will work here? Creating scheduledThreadPool with 5 core threads
 ScheduledExecutorService scheduledThreadPool = Executors.newScheduledThreadPool(5);

Now my application concurrently call scheduledThreadPool for ten tasks at same time
 scheduledThreadPool.schedule(myTask, 0, TimeUnit.SECONDS);

My question is will scheduledThreadPool pick 5 tasks  and complete them first. And then it will pick the rest 5?

A: Short Asnwer
Your shcedulerTreadPool will pick first 5 tasks and put other 5 tasks to queue. Then When one of the currently busy threads exit next task will be scheduled to this thread
In other words
Your schedulerTread pool holds every second exactly 5 threads.Tasks are submitted to the pool via an internal queue, which holds extra tasks(for you everything above 5 tasks currently runing) whenever there are more active tasks than threads.
To asnwer your question


*

*First 5 tasks are picked

*Whenever threads finishes task new task from queue is assigned to it

*Repeat until there are tasks in internal queue 

A: Each task is treated independently and the delay of each task is indenpendant of the task actually being executed.
The delay can be 0 whereas the task has not been started yet because there is no available thread in the pool to execute it.
In your example, the pool will execute the first 5 tasks while the 5 remaining ones wait to be started with a 0 delay. As soon as each thread is done executing a task, it will execute the next one because there is no delay to wait.

A: The order of execution with Threads is undefined. The only thing you can be sure of, is that your tasks will be started in the order you added them. But when they complete and in which order is undefined.
It might be that Thread 1-5 pick task 1-5. Then Thread 4 finishes first and takes task 6, gets interrupted by Thread 2, which finished task 2, but before it can take task 7, it is interrupted by Thread 5, which finishes and takes task 7 first. And so on.
There are some orders of execution which are likely on a single system and setup, but this order might be entirely different or totally chaotic on a different system or even on the same system if you run the program a few times.
