We have released Rspamd 1.9.3 today.

This release contains some new features and many bug fixes. There are no incompatible changes introduced with this release to our best knowledge.

This release includes the following features and important changes.

Hashicorp Vault support

From version 1.9.3, Rspamd can use Hashicorp Vault to store and manage DKIM keys. Vault usage provides secure and flexible storage of the private keys that can scale and use various backends to store sensible data (secrets).

There is a new subcommand for rspamadm utility called vault that is intended to create/remove and securely rotate DKIM private keys using vault.

You can read more about it using the following link: https://rspamd.com/doc/modules/dkim_signing.html#dkim-signing-using-vault

Added least passthrough result

Some modules should set metric result as least possible result. For example, DMARC policy failure should at least mark failed messages as spam but it should not prevent messages from being rejected. From this release, such modules use least policy to set actions allowing to apply a more strict policy if needed.

Tunable memory management

From this version, Rspamd allows to manage memory policies for Lua garbage collection allowing to fit memory/cpu constraints more flexible. When Rspamd is built with jemalloc (e.g. in the default packages provided by the project), it can also print detailed memory statistics on full gc loops.

Here is an example for tuning GC in Rspamd when there are lots of free memory available (around 1Gb per scanner process):

# local.d/options.inc
# http://pgl.yoyo.org/luai/i/2.10+Garbage+Collection
lua_gc_step = 100;
lua_gc_pause = 400;
# number of scanned messages to perform full GC iteration 
full_gc_iters = 10000;

Improved oversigning logic

It is now possible to oversign existing only headers and ignore it if a header is missing. It is done by changing (o) to (x).

The default list of headers signed is changed accordingly:

Important bugs fixes

Here is the list of the most important bugs fixes:

• HTML: Fix size attribute processing - this issue caused rule MANY_INVISIBLE_PARTS to be improperly triggered on many HTML messages
• Do not blacklist mail by SPF/DMARC for local/authed users
• Lots of Clickhouse plugin fixes
• Fix buffer overflow when printing small floats - this issue caused random crashes in WebUI reported by many users
• Fix DoS caused by bug in glib - details in https://gitlab.gnome.org/GNOME/glib/issues/1775

Full list of the meaningful changes

• [Conf] Add IP_SCORE_FREEMAIL composite rule
• [Feature] Add cryptobox method to generate dkim keypairs
• [Feature] Add fast hashes to lua cryptobox hash
• [Feature] Add least passthrough results
• [Feature] Allow oversign if exists mode
• [Feature] Clickhouse: Modernise table initial schema
• [Feature] Implement IUF interface for specific fast hashes
• [Feature] Lua_util: Allow to obfuscate different fields
• [Feature] Tune memory management in Rspamd and Lua
• [Fix] Avoid buffer overflow when printing long lua strings
• [Fix] Change the default oversigning headers to a more sane list
• [Fix] Clickhouse: Do not store digest as it is not needed now
• [Fix] Clickhouse: Fix lots of storage issues
• [Fix] Clickhouse: Support custom actions
• [Fix] Deny URLs where hostname is bogus
• [Fix] Do not blacklist mail by SPF/DMARC for local/authed users
• [Fix] Fix DoS caused by bug in glib
• [Fix] Fix UCL parsing of the multiline strings
• [Fix] Fix buffer overflow when printing small floats
• [Fix] Fix init code for servers keypairs cache
• [Fix] Fix issue with urls with no tld (e.g. IP)
• [Fix] Fix memory in arc signing logic
• [Fix] Fix memory leak in language detector during reloads
• [Fix] Fix mixed case content type processing
• [Fix] Fix processing of the ip urls in file
• [Fix] Fix use after free
• [Fix] HTML: Fix size attribute processing
• [Fix] Hum, it seems that 99ff1c8 was not correct
• [Fix] Lua_task: Fix task:get_from method
• [Fix] Preserve fd when mapping file to scan
• [Fix] Re-use milter_headers settings when doing arc signing
• [Fix] Set dmarc force action as least action
• [Fix] Switch to GMT
• [Fix] allow PKCS7 signatures to be text/plain, too
• [Project] Add initial version of the vault management tool
• [Project] Add vault support for DKIM and ARC signing
• [Project] Implement keys rotation in the vault
• [Project] Improve dkim keys generation for vault
• [Project] Improve keys creation in rspamadm vault
• [Rework] Move lua_worker to a dedicated unit
• [WebUI] Add URL fragments (#) support
• [WebUI] Fix AJAX request URL