In short, I wanted to build a compiler of some C subset that would work at compile-time. It compiles into a custom byte-code for a runtime VM.
I've once tried to write a compile-time C compiler, but I abandoned that project, because I made it overly complex (one-pass compiler right into x86). No clear separation between parser, lexer, etc.
Why would I even want this? Idk. But how can it be useful?
- The code of the compiler doesn't go to the resulting binary,
- No need to waste time for compilation at runtime too,
- Guaranteed type-safety. There can't be such thing as "oh, I changed the function signature, but forgot to update the bindings and it crashed at runtime"
And I shouldn't forget about cons:
- No optimisations. Real compilers spent decades on them and I'm definitely not going to implement LLVM at runtime. Although we could make a compile-time x86 VM, so we can run it at compile time... no, thank you, it's a topic for another fever dream article.
- Hot-reload! I mean, no hot-reload. I won't even mention it anymore, considering that the script is compiled at compile-time and is builtin right into the binary file. I could implement it with hot memory patching or smth, but who really needs it.
Let's start.
## Bypassing constexpr limitations
C++ 20 lets us to dynamically allocate memory at compile-time and even use
std::vector that really expands our borders. But there's one very important note - you can't declare a compile-time vector and extract it into the runtime. No constexpr std::vector<int> data = makeData();, it won't compile. So we need to hack it.### Passing strings in templates
Sadly, the C++ Committee made a lot of cool compile-time features, but not enough (at least for me). We still can't use strings in templates without hacks. But we can easily bypass it with a well-known trick.
template<std::size_t N>
struct const_string {
constexpr const_string() = default;
// implicit-constructor that lets us to do bad things
constexpr const_string(const char (&str)[N]) {
std::copy_n(str, N, value);
}
constexpr operator std::string_view() const {
return {value, value + N - 1};
}
char value[N]{};
const std::size_t length = N;
};
// using it
template<const_string str>
auto very_smart_function(...) { /* ... */ }
### Extracting vectors from compile time
It turned out to be not really that hard, but I didn't really find any ready examples on Internet, unlike with
const_string.To extract
std::vector<T> from constexpr we need to make it std::array<T, N> somehow. The main problem is that we can't write std::array<T, myVector.size()>, because myVector.size() won't be a constant value. So we must to make it constant somehow.I thought of passing vector as a template parameter, but we can't do it legally. C++ 20 allows us to pass only the structs with all-public members. After deeply thinking a bit (not really), I discovered that I could simply pass the lambda that returns our vector (I didn't think I could just pass a pointer actually).
// data_getter is our lambda
template<auto data_getter>
constexpr auto to_array() {
using value_type = typename decltype(data_getter())::value_type;
constexpr static std::size_t size = data_getter().size();
// Create a static array with a "dynamic" size and copy all data
std::array<value_type, size> out;
auto in = data_getter();
for (std::size_t i = 0; i < size; ++i) {
out[i] = in[i];
}
return out; // yay
}
template<const_string str>
constexpr auto lex() {
constexpr static auto data_getter = [] constexpr {
// .lex() returns the vector of tokens
return lexer{static_cast<std::string_view>(str)}.lex();
};
// All our data are available for runtime now =D
return to_array<data_getter>();
}
### Printing errors
For nice errors C++ has
static_assert that allows us to even print our custom message! But it must be always a literal (until C++