If the first thing that came to mind was the popular application that allows you to view content through the torrent protocol, I am afraid to tell you that no, it is not like that, what we are talking about is about developers from Virginia Polytechnic University who recently did a proposal for discussion with Linux kernel developers on a set of patches with the implementation of the Popcorn system (Distributed Thread Execution) to distribute threads.
This system allows you to organize the execution of applications on multiple computers with distribution and migration transparent flow between hosts. With Popcorn, applications can be started on one host, after which they can be transferred to another host without interruption. In multithreaded programs, migration to other single-threaded hosts is allowed.
About Popcorn
Unlike the CRIU project, which allows you to save the process status and resume execution on another system, Popcorn provides a transparent and dynamic migration between hosts during application execution, without requiring user action and ensuring virtual memory consistency on all hosts where the threads run concurrently.
P provides a patch stack to the Linux kernel and the library with a test showing how Popcorn's system calls from migratory flows can be used in a distributed executable applications.
At the kernel level, extensions to the virtual memory subsystem are proposed with the distributed shared memory implementation, which allows processes on different hosts to access a common and consistent virtual address space. The consistency of the virtual memory pages is ensured by a protocol that replicates the memory pages to the host when they are accessed for reading and invalidates the memory pages when written.
Interaction between the hosts is carried out using a kernel-level message handler transmitted over a TCP socket. It is observed that TCP / IP is used to simplify debugging and testing during development. Developers understand that in terms of security and performance, TCP / IP is not the best way to transfer the contents of kernel structures and memory pages between hosts. All hosts that run distributed applications must have the same level of trust. After stabilization of the basic algorithms, a more efficient mode of transport will be applied.
Since 2014, Popcorn has been developed as a research project to study the possibilities of creating distributed applications, whose threads can be executed in different nodes in heterogeneous computer systems, in which the cores can be combined based on various command set architectures (Xeon / Xeon-Phi, ARM / x86, CPU / GPU / FPGA).
The patch set proposed to Linux kernel developers only supports running on hosts with an x86 CPU, But also there is a more functional version of Popcorn Linux which allows applications to run on hosts with different CPU architectures (x86 and ARM).
To use Popcorn in heterogeneous environments, you must use a special LLVM-based compiler. With distributed execution on hosts with the same architecture, rebuilding by a separate compiler is not required.
Also, we can notice the announcement of something similar to the Telefork project with the initial prototype API implementation to start child processes on other computers in the cluster (like fork (), but transfer the branching process to another computer).
The code is written in Rust and so far allows cloning only the simplest processes that do not use system resources, such as files. When making a telefork call, the memory and structures related to the process are cloned to another host running the server controller (telepad).
Using ptrace, the memory mirror of the process is serialized and, along with the process state and logs, transferred to another host. The API also allows you to save the state of a process to a file and restore it through it.
Source: https://lkml.org/