fast_io

fast_io is a new C++20 library for extremely fast input/output and aims to replace iostream and cstdio. It is header-only (module only in the future) for easy inclusion in your project. It requires a capable C++20 compiler supporting concepts.

Hello World

#include<fast_io.h>

int main()
{
	print("Hello World!\n");
}

Compiler Support

• GCC 11. You can download the latest GCC compiler for windows here. https://bitbucket.org/ejsvifq_mabmip/mingw-gcc/src/master/ For Linux, you can watch this video to help you install the latest GCC easily. https://www.youtube.com/watch?v=qFToZjo2x24
• VS 19.26
• Clang 11 with libstdc++. No Clang with libc++ yet since libc++ has not supported concepts and ranges headers.
• Todo: Intel C++ Compiler

Platform Support

• Windows
• Linux
• BSD platforms (Including FreeBSD, NetBSD, OpenBSD and Mac)
• Todo: Webassembly

Supported libc FILE* hacks platforms

• glibc
• MSVCRT
• Universal CRT
• BSD libc
• MUSL libc
• To do: Bionic

Supported C++ standard library std::streambuf/std::filebuf hacks platforms

• GCC libstdc++
• LLVM libc++
• MSVC STL

Design Goal

A general purpose I/O library to replace stdio.h and iostream

• As close to system call as possible.
• No default locale. It is optional.

Safe

• No easily misused stuff like std::endl
• No internal iomanip states (since it creates security issues)
• Providing RAII for C FILE*&POSIX fd&win32 HANDLE
• No dual error reporting mechanism. Exception as the ONLY error reporting mechanism.
• No std::error_code and std::system_error

Easy to use

• Unicode/UTF-8 support
• Support POSIX iconv. You can use fast_io for your coding conversion.
• Compatible with C stdio and C++ iostream
• Binary serialization for trivially copyable types and C++ standard library containers
• All fast_io devices can be natively put in C++ containers. std::vector<fast_io::obuf_file> is valid
• Basic/Lua/Python/etc format (print, scan). No support to C and C++ since they are security hazards.
• Static I/O manipulator
• Provide APIs to expose the internal implementation of C FILE* and C++ stream.

Customizability

• Exception Safe & Exception neutral
• Native Handle Interface
• Extremely easy to support custom devices
• std::mutex mutex stream
• Compilation time open mode parse. Supports C style open mode and C++ style open mode.
• No traits_type and EOF
• Dynamic Type Support
• Multi Process
• Memory map
• Cryptography (Under construction) to replace openssl
• debugging IO
• GUI debugging IO
• Freestanding mode
• Round-trip floating point algorithm
• Network
• Intrinsic SHA-1, Intrinsic HMAC-SHA1, Intrinsic SHA-256, Intrinsic HMAC-SHA256, SHA-512, HMAC-SHA512
• zlib compression/decompression support
• Non crypto hash algorithms support. Jenkins Hash

Post C++20 Plan

1. Module support
2. Coroutine support for async IO
3. Improve and refactor code once Zero-overhead deterministic exceptions are added to the standard

Possible Improvements

2. Interfaces for cloud computing algorithms like MapReduce

I'm aiming for this to be in the C++ standard library in the future :)

Getting Started

Please see examples in the examples folder.

compile option:

g++ -o example example.cc -Ofast -std=c++20 -s

Documentation

See Wiki Page: https://github.com/expnkx/fast_io/wiki

Benchmarks

1. I/O 10M integers

Goal: Print out ten million integers from 0 to 10M to file. Then reopen that file to scan back.

All benchmarks are in benchmarks/0000.10m_size_t/unit.

Notice: I modified libstdc++'s BUFSIZ 1048576 due to BUFSIZE is too small (512 bytes) for MinGW-W64, or it performs horribly.

Run the same test on MSVC 19.26.28805.

Run the same test on GCC 11. glibc + libstdc++

You can see fast_io can also boost the performance of existing facilities for 10x! Yes, it can even improve FILE* and fstream's performance for 10x depending on platforms since I use concepts to abstract them all. fmtlib actually slows down I/O performance. In general, fmtlib and charconv fucking sucks.

2. Output 10M double in round-trip mode with Ryu algorithm

We only perform this test for MSVC since only msvc's charconv implements it. Yes. fast_io defeats msvc's charconv for over 20% for running the same algorithm.

All benchmarks are in benchmarks/0001.10m_double/charconv.

Run the same test on MSVC 19.26.28805.

3. Raw I/O Performance

All benchmarks are in benchmarks/0014.file_io/file_io.

Output 100000000x "Hello World\n"

Notice: I modified libstdc++'s std::filebuf's BUFSIZ to 1048576 due to BUFSIZE is too small (512 bytes) for MinGW-W64 or it performs horribly.

4. Binary Size Just use the benchmark in benchmarks/0014.file_io/file_io. That would introduce Ryu floating table for printing time, and I did not compile the code with -DFAST_IO_OPTIMIZE_SIZE. Or it would be nothing. Dude, you should avoid stream as plague tbh. It is not healthy.

5. Code Convert

Generate 100000000 🚄 emojis by using the program in benchmarks/0020.utf/fill_nc.cc

Benchmarks are in examples/0043.iconv Universal iconv. (UTF-8 to GB18030 as an example) iconv test:

UTF8->UTF16LE

Benchmarks are in examples/0022.utf

iconv test:

UTF8->UTF32LE

Benchmarks are in examples/0022.utf

iconv test: