Melt — Developer Documentation

This guide explains how to develop the Melt language: get the source, compile it, understand the process, and make changes step by step.

1. Prerequisites

C++17 compiler (GCC, Clang, or MSVC)
CMake 3.16 or newer (main build system)
For full build: SDL2, SQLite3, libqrencode (pkg-config). Optional: MySQL client, libzip (for zip extension).
On Windows (MinGW): MSYS2 MINGW64 and g++; no CMake required if using Makefile.win.

Install examples:

Ubuntu/Debian: sudo apt install build-essential cmake pkg-config libsdl2-dev libsqlite3-dev libqrencode-dev
macOS: brew install cmake pkg-config sdl2 sqlite qrencode
Fedora: dnf install cmake gcc-c++ SDL2-devel sqlite-devel qrencode-devel

2. Get the source

Step 1. Clone the repository (or download a release):

git clone https://github.com/melting-language/melting-lang.git
cd melting-lang

All commands below are run from the project root (melting-lang/).

3. Project structure (file by file)

Below is a file-by-file overview of the repository. The compiler pipeline lives in src/core/; optional features are in src/http/, src/sqlite/, etc.

Root and config

File	Role
`src/main.cpp`	Entry point. Parses CLI (script path or no-args help). Reads the script file, creates a Lexer and Parser, gets the AST, then creates the Interpreter, sets the binary directory for extensions, and calls `interpret()`. No language logic here—only orchestration.
`config.h.in`	CMake config template. Contains `#cmakedefine USE_GUI`, `USE_MYSQL`, `USE_SQLITE`, `USE_QR` and `@PROJECT_VERSION_*@`. CMake generates `build/config.h` from this.
`config_win.h`	Windows (Makefile.win) config. Not processed by CMake. Copied to `build/config.h` by Makefile.win so the compiler finds version and `#undef` for USE_* (minimal build without GUI/SQLite/MySQL/QR).

src/core/ — language core

File	Role
`lexer.hpp`	Token definitions. Declares `TokenType` (Number, String, Identifier, keywords like Print, Let, If, Class, etc.) and `struct Token` (type, value, line). Declares class `Lexer` with `tokenize()`.
`lexer.cpp`	Lexer implementation. Reads source character-by-character: `number()`, `string_()`, `identifier()` (and keyword detection), skips whitespace and comments. Produces the token stream consumed by the parser.
`ast.hpp`	Abstract Syntax Tree. Defines all node types: expression nodes (`NumberExpr`, `StringExpr`, `BinaryExpr`, `CallExpr`, `GetExpr`, `ArrayExpr`, `MapExpr`, etc.) and statement nodes (`PrintStmt`, `LetStmt`, `BlockStmt`, `IfStmt`, `ClassStmt`, etc.). No logic—pure data structures.
`parser.hpp`	Parser declaration. Class `Parser` with `parse()` returning a vector of statements. Declares internal helpers: `statement()`, `expression()`, `assignment()`, `term()`, `factor()`, `primary()`, and per-statement parsers (`printStatement()`, `letStatement()`, `classStatement()`, etc.).
`parser.cpp`	Parser implementation. Consumes tokens and builds AST nodes. Recursive descent: `statement()` dispatches to specific statement parsers; `expression()` through `primary()` handle precedence. Parses classes, methods, blocks, if/while/for, imports, try/catch.
`interpreter.hpp`	Interpreter and runtime types. Defines `Value` (variant: number, string, bool, MeltObject, MeltClass, MeltArray, MeltVec, BoundMethod, NativeFunc, MeltClosure), `MeltObject`, `MeltClass`, `MeltMethod`, `MeltArray`, `MeltVec`. Class `Interpreter`: `interpret()`, `execute()`, `evaluate()`, HTTP/MCP hooks, and registration points for built-ins.
`interpreter.cpp`	Interpreter implementation. Walks the AST: `execute(Stmt)` for statements, `evaluate(Expr)` for expressions. Handles variables, assignment, method calls, `this`, classes and instances, returns, imports. Registers all built-ins (print, arrays, JSON, file I/O, HTTP, SQLite, MySQL, QR, MCP, etc.). Largest file in the project.
`module_loader.hpp`	Module loader declaration. Declares `loadExtensions(Interpreter*, binDir, extensionDir, extensionList)` to load shared libraries and call `melt_register(interp)` in each.
`module_loader.cpp`	Module loader implementation. Resolves paths (e.g. `binDir/modules/zip.so`), dlopen/dlsym (or platform equivalent), calls `melt_register`. Used at interpreter startup when `extension` is set in melt.ini / melt.config.

src/http/ — HTTP server

File	Role
`http_server.hpp`	Declares `runHttpServer(Interpreter*, port)`. The server blocks and, for each request, sets request data on the interpreter and calls the handler.
`http_server.cpp`	Implements TCP listen, accept, read request (path, method, body, headers), calls `interp->setRequestData()` and `interp->callHandler()`, then sends the response. Built-ins like `listen(port)` and `setHandler("ClassName")` are registered in interpreter.cpp and call into this.

src/sqlite/ — SQLite built-in

File	Role
`sqlite_builtin.hpp`	Declares the SQLite registration function (called from interpreter when `USE_SQLITE` is defined).
`sqlite_builtin.cpp`	Implements `sqliteOpen`, `sqliteExec`, `sqliteQuery`, `sqliteFetchRow`, etc., and registers them as built-ins. Wraps the SQLite C API.

src/mysql/ — MySQL built-in

File	Role
`mysql_builtin.hpp`	Declares MySQL built-in registration (used only when `USE_MYSQL` is defined).
`mysql_builtin.cpp`	Implements `mysqlConnect`, `mysqlQuery`, `mysqlFetchRow`, `mysqlClose`. Compiled and linked only if CMake finds MySQL client and `USE_MYSQL=ON`.

src/gui/ — image preview

File	Role
`gui_window.hpp`	Declares the GUI window API used by the interpreter to show image previews (e.g. from image built-ins).
`gui_window.cpp`	Implements an SDL2 window that displays a buffer as an image. Used when `USE_GUI` is defined; otherwise the interpreter uses a no-op or stub.

src/qr/ — QR code

File	Role
`qr_builtin.hpp`	Declares QR built-in registration.
`qr_builtin.cpp`	Implements `qrGenerate` (or similar) using libqrencode and registers it. Wired in interpreter.cpp when `USE_QR` is defined.

extensions/

Each subdirectory (e.g. extensions/zip/, extensions/datetime/) contains <name>.cpp and optionally a README. The top-level extensions/CMakeLists.txt discovers them and builds shared libraries into build/modules/. Each extension implements extern "C" void melt_register(Interpreter*); and registers its own built-ins. Examples: zip (ZIP create/extract; needs libzip), datetime (date/time helpers), os, ffmpeg, image_optimize, headless_browser, example.

Other directories

examples/ — Melt scripts and small projects (hello.melt, server, mcp_server_framework, admin_panel_sqlite, etc.).
docs/ — Markdown and HTML documentation (getting started, language reference, built-ins, this dev guide).

Flow: entry point to execution (how the files work together)

When you run ./build/melt script.melt, execution follows this path. Each step passes data to the next until the script finishes (or starts an HTTP/MCP server and blocks).

main.cpp → 1. Read script file, get source string

Lexer (lexer.cpp) → 2. Tokenize: source → list of Token

Parser (parser.cpp) → 3. Parse: tokens → AST (vector of Stmt)

Interpreter (interpreter.cpp) → 4. interpret(): load extensions, then execute each Stmt

execute() / evaluate() → 5. Walk AST: run statements, evaluate expressions, call built-ins

(optional) HTTP / MCP → 6. If script called listen() or runMcp(), process runs until exit

Step-by-step:

main.cpp — Entry point. Reads script.melt into a string. If no argument, prints usage and exits.
Lexer — Lexer lexer(source); auto tokens = lexer.tokenize(); turns the source into a stream of tokens (keywords, identifiers, numbers, strings, operators). Uses lexer.cpp.
Parser — Parser parser(tokens, path); auto statements = parser.parse(); consumes tokens and builds an Abstract Syntax Tree (a list of statement nodes). Uses parser.cpp and types from ast.hpp.
Interpreter — Interpreter interpreter; interpreter.setBinDir(binDir); interpreter.interpret(statements, entryPath); First, the interpreter loads extensions (module_loader) if configured, and registers all built-ins (from interpreter.cpp and optional sqlite/mysql/qr/http). Then it runs interpret().
Execution — interpret() loops over each top-level statement and calls execute(stmt). For each statement type (e.g. PrintStmt, LetStmt, IfStmt), it runs the corresponding logic; for expressions it calls evaluate(expr). Variables are stored in the interpreter’s scope; method calls resolve to Melt methods or to registered built-ins (e.g. print, jsonEncode). Imports load another file and run its AST in the same interpreter.
End or long-running — When the top-level statements finish, interpret() returns and main exits. If the script called listen(port) or runMcp(), those built-ins block inside the interpreter (HTTP server loop or MCP stdio loop), so the process keeps running until the server is stopped or stdin closes.

So the data flow is: source string → tokens → AST → execution. The entry point is main.cpp; the “last” step is either normal exit after interpret() returns or process exit after a long-running server stops.

Memory management

Melt does not use a garbage collector. Memory is managed with C++ RAII and smart pointers.

Summary: The AST is owned exclusively by the parser/caller (unique_ptr) and lives until the end of interpret(). Runtime values are either value types (number, string, bool) or shared_ptr to heap data; when the last reference goes away, the object is destroyed. There is no separate GC pass.

Multi-thread management

The interpreter is single-threaded: one Interpreter instance, one variables_ map, no mutex. Script execution and built-ins (including sleep) run on the same thread. This keeps the core simple and avoids data races.

Where threads (or processes) appear:

HTTP server (Windows) — Two threads: the main thread accepts connections and pushes (client, request) into a queue; a single worker thread pops from the queue and runs handleOneRequest(interp, ...). Only the worker touches the interpreter, so requests are handled one at a time. The queue is protected by std::mutex and std::condition_variable (producer–consumer).
HTTP server (Unix) — No threads. The main process fork()s a child per request; the child runs handleOneRequest then exits. The interpreter is not shared across processes.
MCP (stdio) — Single-threaded: one loop reads a line from stdin, calls the handler, writes the response. No worker threads.

Implications:

Melt scripts never run in parallel on the same interpreter. No need for locks in script code or in the interpreter core.
On Windows, listen(port) uses one worker; concurrent requests wait in the queue and are processed sequentially.
Built-ins like sleep(seconds) block the current thread (the one running the script or the HTTP worker).

4. How to compile

Option A: CMake (Linux, macOS, Windows with CMake)

Step 1. Configure (creates build/ and generates config.h):

cmake -B build

Step 2. Build:

cmake --build build

Output: build/melt (or build/melt.exe on Windows). Extensions are built into build/modules/ (e.g. zip.so, datetime.dylib).

Option B: Makefile wrapper (Unix)

The top-level Makefile just calls CMake. No standalone “make compile” of C++; the real build is CMake.

make          # same as: cmake -B build && cmake --build build
make install  # cmake --install build
make clean    # rm -rf build

Option C: Windows without CMake (Makefile.win)

For MinGW (e.g. MSYS2 MINGW64) when you don’t want to use CMake:

Step 1. Generate config.h (done automatically by the Makefile):
build/config.h is created by copying config_win.h into build/.

Step 2. Build:

make -f Makefile.win

Output: bin/melt.exe. Requires config_win.h in the repo (defines version and disables USE_GUI, USE_MYSQL, USE_SQLITE, USE_QR for a minimal build).

Optional: enable MySQL

cmake -B build -DUSE_MYSQL=ON
cmake --build build

5. How Melt works (pipeline)

When you run ./build/melt script.melt, the following happens:

Load script — main.cpp reads the file and passes it to the interpreter.
Lexer — Source text is turned into tokens (keywords, identifiers, numbers, strings, operators). See src/core/lexer.cpp.
Parser — Tokens are turned into an AST (Abstract Syntax Tree). See src/core/parser.cpp and ast.hpp.
Interpreter — The AST is executed (evaluate expressions, run statements, call methods). See src/core/interpreter.cpp.

So the pipeline is: Source → Lexer (tokens) → Parser (AST) → Interpreter (execution).

Built-ins (e.g. print, jsonEncode, listen) are registered in the interpreter and called when the AST references them.

6. Development process

To add or change language features, you usually touch four places:

Step	Where	What to do
1	`lexer.hpp`, `lexer.cpp`	Add token type and recognize new keyword/symbol.
2	`ast.hpp`	Add a struct for the new statement or expression.
3	`parser.hpp`, `parser.cpp`	Parse the new construct and return the new AST node.
4	`interpreter.hpp`, `interpreter.cpp`	Handle the new AST node in `execute()` or `evaluate()` and implement behavior.

Tutorial: For a full walkthrough, see Tutorial: Implement syntax (add say expr; step by step). Reference: 06_DEVELOPMENT_ADDING_SYNTAX.md. After editing:

cmake --build build
./build/melt test.melt

7. Extensions

Extensions are shared libraries that register extra built-ins. They live in extensions/<name>/<name>.cpp and export melt_register(Interpreter*).

Built by CMake into build/modules/ (e.g. zip.so, datetime.dylib).
Enabled in melt.ini or melt.config: extension = zip (or comma-separated list).

To add a new extension: add a directory under extensions/ with <name>.cpp; the top-level extensions/CMakeLists.txt picks up subdirectories that contain a matching .cpp. See extensions/example/ and extensions/README.md-style docs in each extension folder.

8. Config and build options

config.h — Generated by CMake from config.h.in (or from config_win.h on Windows with Makefile.win). Defines USE_GUI, USE_MYSQL, USE_SQLITE, USE_QR and version numbers.
melt.ini / melt.config — Runtime config: modulePath, extension, extension_dir. See docs/03_BUILTINS.md.

9. Test and run

From project root:

./build/melt examples/hello.melt
./build/melt examples/mcp_server_framework/main.melt   # MCP server

Run with no arguments to execute the built-in demo:

./build/melt

10. Release build

CMake: cmake -B build -DCMAKE_BUILD_TYPE=Release then cmake --build build.
CI: GitHub Actions (see .github/workflows/release.yml) build on tag push (e.g. v1.4.0) with CMake on Linux/macOS and Makefile.win on Windows; binaries and Debian package are attached to the release.
When bumping the version, update CMakeLists.txt (project(melt VERSION x.y.z)) and config_win.h (MAJOR, MINOR, PATCH) so Windows Makefile.win stays in sync.

Melt developer documentation. See also docs/README.md, Getting Started, Adding syntax.