OrioleDB combines the advantages of on-disk and in-memory engines, increasing database performance and reducing costs
Recently, the news of the publication of a new storage engine for PostgreSQL called "OrioleDB", which was created by OrioleData.
The goal from OrioleDB is provide fast and compact storageo not cause an overload of the database files over time and that does not require the use of a VACUUM process which runs periodically for garbage collection.
It does this by implementing undo records at the row and block level., as well as automatic page merging. Row and block level undo logs provide a more granular level of control, allowing for more efficient handling of data changes. The automatic page merging feature works tirelessly in the background to consolidate fragmented data, further improving system efficiency.
About OrioleDB
As we already mentioned, OrioleDB is a new engine for PostgreSQL, developed with one main goal: to prevent tables from being overloaded and to eliminate the need for regular maintenance like VACUUM.
The structures used in OrioleDB allow operations to be processed more efficiently of data modification in the database and replace the freed blocks without accumulating them in time. The ability to bypass VACUUM is implemented through the use of rollback logs (undo logs) that work at the block and individual line level, as well as a system for automatically merging data pages.
The row-level rollback record allows for updates with data replacement in place, without releasing the current record and creating a new one. Block-level rollback logging allows you to preempt tuples and free up storage space immediately after a DELETE operation, even though they remain visible in failed transactions. To reduce storage fragmentation that results from a large number of deletions, individual data pages are automatically merged in the background.
The WAL transaction log in OrioleDB is kept at the row level, not in blocks, which makes it easy to use in distributed systems and allows parallelizing logging deployment. It is possible to create distributed configurations with several active master servers, to keep them in a synchronized state the Raft consensus algorithm is used.
Of the characteristics that stand out from OrioleDB, you can see the reading data pages without using locks, direct binding of pages in RAM with pages in permanent storage, the use of the CoW (copy on write) mechanism by setting checkpoints to create consistent snapshots at any time. The 64-bit values are used for transaction identifiers, the use of which solves the problem of counter overflows.
Using OrioleDB allows not only eliminate the need for manual operations and simplify DBMS maintenance, but also Increase performance. For example, better performance is achieved by updating data in place, which avoids changing indexes that are not related to storing the changed value. The engine also supports storing data in a compressed form (compression is performed using the ZSTD algorithm and is implemented at the variable-size block level), which allows you to reduce the size of the database on disk by 4-5 times.
Testing a single table setup with 5 indexes using OrioleDB showed 5x increase in TPS (transactions processed per second), 2.3x reduction in CPU load, 22x reduction in I/O operations per transaction.
Finally if you are interested in knowing more about it, you should know that the engine is written in C and It is distributed under a PostgreSQL license and is implemented as a plugin that requires changes to the core PostgreSQL codebase.
It should be noted that in its current form, the engine is in the beta testing stage and is not yet recommended for production deployments.
You can check the details of the original publication at the following link.