Written on 2023-11-12
Existing literature:
Imagine a situation where there are many separate tasks to be completed - each of which require a certain amount of work or computation. Additionally, they each have an externally bounded start and end point, for example a user connecting. Furthermore, there are time constraints that dictate when all computations must be completed. Thus, the question is how can these tasks be planned and executed?
When using an operating system, a limited number of CPUs and cores are available which are used to distribute the tasks between processes. This can be done either in a cooperative scheduling fashion, in which processes are swapped upon signals received from the processes, or a pre-emptive scheduling fashion in which processes are swapped determined by an operating system priority level.
When using Rust, however, the scheduling must be done at the application level no matter what type of scheduling is used. To optimize this scheduling, computations can be divided into IO-bound and non-IO-bound tasks. IO-bound tasks are bound by external factors, such as data being read or written, while non-IO-bound tasks only depend on local factors and can be handled by the processor more efficiently. Most languages will then spawn separate threads for IO-bound operations and use an event loop to handle and distribute actions accordingly.
The simplest form of computation is synchronous, where each task runs after the previous one in a linear fashion. Synchronous code can be written in either a blocking or non-blocking way. In synchronous blocking code, everything runs on a single thread and each step of the program waits for the completion of the particular function or operation. In comparison, in non-blocking code, the program would be paused at points where the execution would block and then continue elsewhere, where progress can be made. For examples of this code, see languages such as Go and Ruby.
When a language offers ways to schedule events and triggers responses from these events, it is considered asynchronous. In this type of programming, computations run as triggers to some event that arises in the event loop.
In an asynchronous and non-blocking language, events and triggers are usually exposed through an event loop that is continually run. This requires something to be running the loop in order to schedule certain tasks, which is referred to as a runtime and can add some overhead. To avoid this overhead, some languages such as Rust use minimal run-times.
In Rust, every asynchronous function or part is compiled into a state machine that functions similarly to an event loop.
To turn asynchronous functions into state machines, they need to have the ability to pause evaluation or computation at different steps. This process of pausing and resuming is also known as a coroutine, and when it pauses, it yields an intermediate result so that another computation can begin. An example of this would be if a function requires a filled buffer, but does not have it available yet; instead of stalling, it can pause and yield its current execution state while waiting for data.
In JavaScript, computations that can be paused are called promises. These promises are usually evaluated eagerly.
The cornerstone of asynchronous programming in Rust is the Future. A future is also referred to as “promise” in JavaScript, and it is only evaluated when pushed towards completion. Put simply, a future is a planned, potential finished computation, with the possibility of being completed. Examples of futures include the content of a packet when a network packet arrives, or the buffer content to be read when a device writes to a COM port.
A future is implemented as an object that has a poll method. The interface or trait for a future is:
pub trait Future {
type Output;
fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output>;
}Given a future, calling the poll-method is also called polling the future.This method can be used to poll and when the future object is polled, it returns an enum Ready or Pending (when the future is not ready).
enum Poll<T> {
Ready(T),
Pending
}To poll futures, it is not necessary to use the poll method directly. It is actually not recommended.
Instead, there is some syntactic sugar called async/await that allows a more subtle way to poll futures. Async-await is a specific way to write asynchronous functions, functions that have as output a future. During the execution futures may appear as temporary local variables. These can be awaited.
async fn example() {
tokio::time::sleep(Duration::from_millis(50)).await
}When a future is awaited by calling .await, the poll method of the associated future is called. If the poll method returns Pending, the underlying future is not ready and the stack frame of the function is essentially paused. This means the function is freezed until further notice. The function that can be paused is a coroutine. When coroutines pause, we say that the coroutine yields.
An asynchronous function that is blocked temporarily, yields a stack frame. A stack frame is a snapshot of all the local variables and references that are valid at the point in time right before the await. This means that there may be self-references inside of the stack frame. This implies that this future may not be moved, because moving self-referential data in memory can create an invalid object. To prevent this the type of the future in the poll method is Pin<&mut Self>.
See chapter in Rust book.
The previous gives rise to a new way of operating on futures which are essentially computations that may happen in the future. Futures can be joined or selected. Joined futures are all started simultaneously and the joined future completes when each future completes. In a select, the first future to finished finishes the whole future.