fix(rt): bring back dynamic machines

[dignifiedquire]

Even if we do not make use of the progress blocking, we do need to make use of the dynamic restarting of machines as far as I understand.

Keeps the perf, while removing the regression from #747

[yoshuawuyts]

Is perhaps crossbeam utils BackOff a better fit here than rolling it by hand?

[dignifiedquire]

I figured if it was good enough for @stjepang it'll be good enough for me for now ;)

[k-nasa]

@dignifiedquire
What is the cost of starting a thread?
The reason I removed the mechanism to increase or decrease the number of threads is to temporarily omit those discussions.

I think it's good to start a thread when you need it and close it when it's no longer needed.

[win-t]

@dignifiedquire
What is the cost of starting a thread?
The reason I removed the mechanism to increase or decrease the number of threads is to temporarily omit those discussions.

I think it's good to start a thread when you need it and close it when it's no longer needed.

can we cache the thread? make the thread is just park instead of terminating it, then when a new thread is needed just unpark it and assume this thread is a fresh one and make sure no resource leak to the new thread (reinitialize all TLS), something like this

loop {
  main_loop();
  invalidate_tls();
  if push_to_cache(thread::current()) {
    thread::park();
  } else {
    break;
  }
}

[dignifiedquire]

I agree we should investigate parking, but for now this is the original logic and fixes the brokeness on master