Rspamd 1.8.2 has been released

19 Nov 2018

We have released Rspamd 1.8.2 today.

There are no incompatible changes introduced with this version to our best knowledge. However, due to Bayes changes, you might see slightly different Bayes outcomes. rspamd_update module is now disabled by default as we are working on a better and safer solution that will be available in the next Rspamd version.

The most important features and fixes

  • Critical fixes in SURBL module

    SURBL module could insert absurd scores and skip group symbol limit in some cases.

  • Critical fixes in Bayes statistics

    Bayes algorithm has been changed significantly in this release:

    • If chi-squared method fails due to exp underflow, Rspamd will switch to a naive method
    • Tokens weighting has been reworked to match chi-squared model
    • OSB weights have been adjusted to discriminate more far tokens and increase weight for closer tokens
    • Reworked meta-tokens algorithm, it is now moved to Lua
    • Skip stop words in statistics

  • Fuzzy check rework

    In this version, we have moved selection algorithm from C to Lua with more flexibility about parts and data selection.

  • Critical fixes in Ratelimit module

    We have found and fixed various issues in Ratelimit plugin and restored both traditional and modern syntax of ratelimits. Presumably, all version prior this one are broken when dealing with ratelimits.

  • Major rework of all asynchronous rules and events

    In Rspamd 1.8.2, we have introduced new model of asynchronous rules processing with the following features:

    • More straightforward in terms of the control flow
    • It is possible to get better debugging info in case of issues (including the concrete line of code)
    • It is possible now to see the real execution time of rules and show slow rules including network waiting
    • New model is less complex than the previous one

  • Critical issue in ARC module

    We have fixed the issue in ARC module that could lead to around 11% of wrong signatures being produced due to an incorrect \0 handling in raw RSA signature.

  • Add support for encrypted cookies in Message-ID

    It is now possible to generate and check message-ids that require no storage and Rspamd even don’t need to observe outbound messages. However, it will be able to detect replies to our messages and get some more information if needed:

    We generate message id using nonce||aesencrypt(nonce, k, known_cookie) then we can just take a look into In-Reply-To header and apply read_nonce() + aesdecrypt(nonce, k) to compare result with known_cookie.

    Nonce length will be 16 bytes (32 hex encoded) and another 16 bytes could be used to encrypt secret cookie + some random thing derived from the message itself.

  • Added DWL support in the default configuration
  • Add support of words regexps
  • Allow to set custom limits for upstream lists
  • Detect orphaned parts and attach them to message
  • Support query arguments in controller’s custom commands
  • Write DKIM selector in dkim allow/reject symbols

Full list of the meaningful changes

  • [Conf] Add DWL support in the default configuration
  • [Conf] Disable rspamd_update by default (again)
  • [Conf] Fix configuration sample for ratelimit
  • [CritFix] Disable broken url tags by default
  • [CritFix] Fix \0 processing when doing RSA sign
  • [CritFix] Fix adding symbols to their primary groups
  • [Feature] Add rspamadm cookie utility
  • [Feature] Add specialised functions for generating encrypted cookies
  • [Feature] Add support of cookies in replies module
  • [Feature] Add support of words regexps
  • [Feature] Allow to add 3rd party clang plugins
  • [Feature] Allow to create lua regexps from glob or plain patterns
  • [Feature] Allow to set custom limits for upstream lists
  • [Feature] Detect orphaned parts and attach them to message
  • [Feature] Filter tokens in bayes
  • [Feature] Fold b= value when doing arc sealing
  • [Feature] Ignore cookies in the future and too old in the past
  • [Feature] Skip stop words in statistics
  • [Feature] Store stop words and allow to query them
  • [Feature] Support query arguments in controller’s custom commands
  • [Feature] Tune upstream limits in Rspamd proxy
  • [Feature] Use different callback symbols for different uribls
  • [Feature] Write DKIM selector in dkim allow/reject symbols
  • [Fix] Add obs_fws state support to eoh state machine
  • [Fix] Add sanity check when applying mime boundaries heuristic
  • [Fix] Antivirus - virus names with 0 were recognized as tables
  • [Fix] Disable headernames in bayes temporarily
  • [Fix] Do not allow syntax errors in include files…
  • [Fix] Do not allow to merge an object with an array (or vice versa)
  • [Fix] Don’t perform forged recipients check for missing recipients
  • [Fix] Fix DKIM based RBLs
  • [Fix] Fix actrie implementation (sync from upstream), fixed OOB read
  • [Fix] Fix explicit methods call in selectors
  • [Fix] Fix extraction of additional parts
  • [Fix] Fix finalization for internal plugins
  • [Fix] Fix override_defaults function
  • [Fix] Fix squeezed symbols when using settings
  • [Fix] Fix urls insertion in Clickhouse module
  • [Fix] Furhter fixes to ratelimits logic
  • [Fix] Ignore signatures when looking for boundaries
  • [Fix] Properly set learned count
  • [Fix] Really fix ratelimits configuration and work
  • [Fix] Remove ambigious format flag from printf
  • [Fix] Restore URLs exporting in ClickHouse plugin
  • [Fix] Rework bayes calculations…
  • [Fix] Switch from chi-square to naive for large Fisher value
  • [Fix] Treat normal password as enable password if there is no enable password
  • [Fix] Use proper syntax for making DNS requests
  • [Fix] Various fixes in embedded plugins
  • [Project] Change fuzzy check selection logic to lua_fuzzy library
  • [Project] Rework async events and symbols
  • [Project] Move all metatokens in Bayes to lua_stat from C
  • [WebUI] Add history rows per page control