The byte compare feature in logical signatures will cause the rule to
alert if it successfully matches regardless of the rest of the logical
signature.
An easy way to test this is with a logical signature that has two
bcomp subsignatures and requires both to match for the rule to alert.
In the following example, we have 4 signatures where
- the first will match both bcomp subsigs.
- the second will match neither.
- the last two match just one bcomp subsig.
In an --allmatch test, you'll find that the 3 of these match, with the
first one matching *twice*, once for each bcomp subsig.
test.ldb:
```
bcomp.both;Engine:51-255,Target:0;0&1&2&3;4141;0(>>5#hb2#=123);4242;2(>>5#hb2#=255)
bcomp.neither;Engine:51-255,Target:0;0&1&2&3;4141;0(>>5#hb2#=124);4242;2(>>5#hb2#=254)
bcomp.second;Engine:51-255,Target:0;0&1&2&3;4141;0(>>5#hb2#=124);4242;2(>>5#hb2#=255)
bcomp.first;Engine:51-255,Target:0;0&1&2&3;4141;0(>>5#hb2#=123);4242;2(>>5#hb2#=254)
```
test.sample:
```
AA = 7B; BB = FF
```
You can also try a similar test to compare the behavior with regular
ac-pattern-match subsigs with this lsig-test.ldb:
```
pattern.both;Engine:51-255,Target:0;0&1;4141;4242
pattern.neither;Engine:51-255,Target:0;0&1;4140;4241
pattern.second;Engine:51-255,Target:0;0&1;4140;4242
pattern.first;Engine:51-255,Target:0;0&1;4141;4241
```
This commit fixes the issue by incrementing the logical subsignature
count for each bcomp subsig match instead of appending an alert for
each bcomp match.
Also removed call to `lsig_sub_matched()` that didn't do anything.
* Added loglevel parameter to logg()
* Fix logg and mprintf internals with new loglevels
* Update all logg calls to set loglevel
* Update all mprintf calls to set loglevel
* Fix hidden logg calls
* Executed clam-format
Add support for compiling with external TomsFastMath library provided by
the system instead of compiling the vendored copy into libclamav.
The vendored source is still built directly into libclamav instead of as
a separate library the way libmspack is done.
The rationale is that:
A) it's more complicated to deal with possibly compiling as static or
dynamic, and also
B) libmspack and libunrar are compiled separately primarily because of
licensing concerns. TomsFastMath public domain, so that isn't a concern.
Resolves: https://bugzilla.clamav.net/show_bug.cgi?id=12562
Update the vendored TomsFastMath (TFM) library to v0.13.1.
Resolves: https://bugzilla.clamav.net/show_bug.cgi?id=11992
I removed compatibility macro's from when libTomMath was used.
This required removing a bunch of faux-error handling because
the fast-math equivalent functions return void, and cannot fail.
The previous version used had named the header "bignum_fast.h"
instead of "tfm.h" and had customizations in that header to enable
TFM_CHECK all the time, and also TFM_NO_ASM if __GNUC__ not defined
or if the system isn't 64bit architecture. This update uses tfm.h
as-is, and has CMake define TFM_CHECK and TFM_NO_ASM as needed.
I've kept bignum.h as an interface to including tfm.h so that in
the future we can more easily add support for system-installed
TomsFastMath instead of the vendored one, taking inspiration from
Debian's patch to support system-TomsFastMath.
See: https://salsa.debian.org/clamav-team/clamav/-/blob/unstable/debian/patches/add-support-for-system-tomsfastmath.patch
* Work with data as &[u8] instead String/&str to avoid unnecessary UTF-8
validation and reuse read buffers.
* Make error handling more concise
* Address Clippy-raised issues
The logging functions use a callback to print log messages. Because the
callback could be anything provided by an application, it isn't safe to
log while holding a mutex.
This commit defers error reporting in cacheset_add() to prevent running
the callback while the mutex is held.
The CIFuzz tool was observed detecting an issue in code unrelated to a
pull request. While it was relatively benign in this case, it has the
potential to accidentally disclose a vulnerability publicly without
allowing a non-disclosure period for us to publish a fixed patch
version.
I'm not comfortable with this risk, so I'm removing the workflow.
RelWithDebInfo is our preferred build type. It has optimizations and
should run faster, but includes debugging symbols for better profiling,
stack traces, etc.
The Rust MinSizeRel support is just Release mode for now. There are
optimizations we can do to shrink it further, but for now at least it
won't actually be Debug (aka slow).
I found two issues with the cl_load fuzz targets, one of which impacts
the scanfile and scanmap fuzz targets:
1. We were defining the preprocessor definitions incorrectly using
"SCAN_TARGETS" instead of "TARGET" in unit_tests/CMakeLists.txt.
For the scan fuzz targets this meant it wasn't properly defining
unique settings for each compiled target.
For the cl_load fuzz target it's worse, it wasn't setting the
database file name correctly which means it rejected the filenames
entirely for not having a legitimate suffix.
2. We were pre-compiling the engine before loading signatures.
You can't load sigs for an engine that's already compiled, so this
would also fail right away without trying to load any sigs.
Resolves https://bugzilla.clamav.net/show_bug.cgi?id=12592
A null-dereference crash happens if the database directory contains:
a good `daily.cvd` symlink
a broken `daily.cld` symlink
or:
a broken `daily.cvd` symlink
a good `daily.cld` symlink
You're not supposed to have both a .cvd and .cld for the same file
at the same time. And making individual symlinks for the database
files is also unexpected.
That is to say that this is not an intended or supported use case
for the database directory. But it's a fairly simple bug to fix.
The issue is that a bad symlink is still added to the database load-
list after the access check fails. This commit skips daily databases
that fail the access check.
The database update script has a line to delete freshclam.dat after the
update, but this fails when attempting to update older images that have
mirrors.dat or no dat file at all. This issue is compounded by a bug
where the -t (--tags) option doesn't work so it tries to update all
images every time, and then of course fails on the older ones.
This commit has the script try removing freshclam.dat or mirrors.dat and
allows it to succeed even if neither exist.
Adds a clamscan test to verify that --allmatch mode works for the
clam.exe program when loading MD5, SHA1, SHA256, PE-section, & PE-import
hash signatures.
The PE section hash scanning code didn't implement the all-match check.
While this check isn't the ideal implementation for all-match mode...
(see the commit message for the previous commit)
...it's simple enough to add the all-match check here for now.
All-match mode is presently not working if you have multiple hash-based
signatures for the same file. That is, only the first signature will
alert before moving on to other parts of the scan.
Our current all-match implementation adds checked into every file parser
(often at multiple layers in the scanning process) to make sure that the
scan continued even if an alert is found, but only when in all-match mode.
This is clearly an error-prone approach.
Instead, I would like to change it so that every alert appends to an alert
list. When in all-match mode, the scan status would continue with "clean"
until the very end when the process is complete and the list of alerts is
evaluated.
Unfortunately, that's a much bigger job. For this specific bug, it is
simple enough to fix the broken logic that aborts the hash scanning
early. In the future, we should rework all of this logic throughout
libclamav, as described above.
In testing, I found that libclang.so/clang.dll is required by bindgen
and was not found on all of our test machines. To resolve this we will
only generate sys.rs bindings when CMake MAINTAINER_MODE option is "ON".
This is unfortunate that we have to commit generated source to version
control. But as a benefit it makes rust-analyzer happier when editing a
workspace that hasn't yet been compiled. And it makes it more reasonable
that the generated sys.rs file generated to the source directory and not
the build directory (something we hadn't resolved yet).
Use bindgen to generate Rust-bindings for some libclamav internal
functions and structures in an new "sys.rs" module.
"sys.rs" will be generated at build-time and unfortunately must be
dropped in the source/libclamav_rust/src directory rather than under the
build directory. As far as I know, Cargo/Rust provide no way to set an
include path to the build directory to separately place generated files.
TODO: Verify if this is true and move sys.rs to the build directory if
possible.
Using the new bindings with:
- the logging module.
- the cdiff module.
Also:
- Removed clamav_rust.h from .gitignore, because we generate it in the
build directory now.
- Removed the hand-written bindings from the cbindgen exclusions.
lib.rs has an annotation that prevents cbindgen from looking at sys.rs.
- Fixed a `u8` -> `c_char` type issue in cdiff in the cli_getdsig() call
parameters.
We received a high FP report rate for Heuristics.PNG.CVE-2010-1205.
This is a 11 year old CVE and is likely no longer relevant anyways.
Removing Heuristics.PNG.CVE-2010-1205 means there is no longer any point
decompressing the PNG image data in the PNG CVE checker. Removing image
data decompression in the CVE checker should improve PNG image scan
times.
The OOXML parser in libclamav may try to extract an entry that is
missing a file name. This results in an invalid 0x1 pointer dereference
in the ZIP parser that is likely to crash the scanning application.
This commit fixes the issue by requiring both the PartName (PN) *and*
the ContentType (CT) variables to be non-NULL or else the entry will be
skipped.
Thank you Laurent Delosieres for reporting this issue.
Rustify cdiff_apply() and clean up error handling:
- Restore [some] safety and clean up error handling.
- Use rust-crypto sha2 instead of OpenSSL's.
Fix signedness of cli_versig2() dsig parameter.
c_char may be an i8 or u8 depending on platform:
https://doc.rust-lang.org/src/std/os/raw/mod.rs.html#91-133
Rustify cmd_close():
- Consolidate DEL/XCHG records.
- Tidy up ADD handling.
- Various error handling cleanup, etc.
Remove some extra clones that don't seem to be neceesary.
Replace writeln format macro with plain write command.
Use a buffered writer for deletes, exchanges, & writes.
Switching from individual `write` syscalls per change to a
buffered writer appears to speed up cdiff-apply by about 2x.
Apply both .cdiff and .script CVD patches.
Note: A script is a non-compressed and unsigned file containing cdiff
commands. There is no header or footer that should be processed.
This Rust-based implementation of the cdiff-apply feature includes
equivalent features as found in the C-based implementation:
- cdiff file signature validation against sha256 of the file contents
- Gz decoding of file contents
- File open command
- File close command
- Signature add command
- Line delete command
- Xchg command
- Move command
- Unlink command
This Rust implementation adds cdiff-apply unit tests to verify correct
functionality.
ClamAV's option parser, used for `freshclam.conf`, `clamd.conf`,
and `clamav-milter.conf` has a max line length of 512 characters.
By request, this commit increases the line length to 1024 to accommodate
very long `DatabaseMirror` options when using access tokens in the URI.
Resolves: https://github.com/Cisco-Talos/clamav/issues/281
You should be able to disable the maxfilesize limit by setting it to
zero. When "disabled", ClamAV should defer to inherent limitations, which
at this time is INT_MAX - 2 bytes.
This works okay for ClamScan and ClamD because our option parser
converts max-filesize=0 to 4294967295 (4GB). But it is presently broken
for other applications using the libclamav C API, like this:
```c
cl_engine_set_num(engine, CL_ENGINE_MAX_FILESIZE, 0);
```
The limit checks added for cl_scanmap_callback and cl_scanfile_callback
in 0.103.4 and 0.104.1 broke this ability because we forgot to check if
the `maxfilesize > 0` before enforcing it.
This commit adds that guard so you can disable by setting to `0`.
While working on this, I also found that the `max_size` variables in our
libmspack scanner code are using an `off_t` type, which is a SIGNED integer
that may be 32bit width even on some 64bit platforms, or may be a 64bit
width. AND the default `max_size` when `maxfilesize == 0` was being set to
UINT_MAX (0xffffffff), aka `-1` when `off_t` is 32bits.
This commit addresses this related issue by:
- changing the `max_size` to use `uint64_t`, like our other limits.
- verifying that `maxfilesize > 0` before using it.
- checking that using `UINT32_MAX` as a backup will not exceed the
max-scansize in the same way that we do with the maxfilesize.
The max bytes supplied to strftime should be the length of result
string, including the terminating null byte.
Without the extra byte for the terminating null byte, the output is one
byte too long and results in undefined behavior:
If the length of the result string (including the terminating null
byte) would exceed max bytes, then strftime() returns 0, and the
contents of the array are undefined.
Also resolve alleged uninitialized memory use by initializing the
`digest` variable in `cli_md5buff()`. MSAN blames it for putting
uninitialized data in the `name_salt` global, though in debugging and in
review I can't find any evidence that it isn't initialized by the call
to `cl_hash_data()` in `cli_md5buff()`.
This MSAN complaint has been a blocker to enabling MSAN in OSS-Fuzz.
Add a basic unit test for the new libclamav_rust `logging.rs` module.
This test simply initializes logging and then prints out a message with
each of the `log` macros.
Also set the Rust edition to 2018 because the default is the 2015
edition in which using external crates is very clunky.
For the Rust test support in CMake this commit adds support for
cross-compiling the Rust tests.
Rust tests must be built for the same LLVM triple (target platform) as
the rest of the project. In particular this is needed to build both
x64 and x86 packages on a 64bit Windows host.
For Alpine, we observed that the LLVM triple for the host platform tools
may be either:
- x86_64-unknown-linux-musl, or
- x86_64-alpine-linux-musl
To support it either way, we look up the host triple with `rustc -vV`
and use that if the musl libc exists. This is a big hacky and
unfortunately means that we probably can't cross-compile to other
platforms when running on a musl libc host. There are probably
improvements to be made to improve cross compiling support.
The Rust test programs must link with libclamav, libclammspack, and
possibly libclamunrar_iface and libclamunrar plus all of the library
dependencies for those libraries.
To do this, we pass the path of each library in environment variables
when building the libclamav_rust unit test program.
Within `libclamav_rust/build.rs`, we read those environment variables.
If set, we parse each into library path and name components to use
as directives for how to build the unit test program.
See: https://doc.rust-lang.org/cargo/reference/build-scripts.html
Our `build.rs` file ignores the library path environment variables if
thye're not set, which is necessary when building the libclamav_rust
library and when libclamunrar isn't static and for when not linking with
a libiconv external to libc.
Rust test programs are built and executed in subdirectory under:
<target>/<llvm triple>/<config>/deps
where "target" for libclamav_rust tests is set to <build>/unit_tests
For example:
clamav/build/unit_tests/x86_64-pc-windows-msvc/debug/deps/clamav_rust-7e1343f8a2bff1cc.exe
Since this program isn't co-located with the rest of the libraries
we also have to set environment variables so the test program can find and
load the shared libraries:
- Windows: PATH
- macOS: DYLD_LIBRARY_PATH
We already set LD_LIBRARY_PATH when not Windows for similar reasons.
Note: In build.rs, we iterate references to LIB_ENV_LINK & Co because
older Rust versions do implement Iterator for [&str].
Add a top-level Cargo.toml.
Remove the vestigial libclamav_rust/Makefile.am.
Place Rust source under a libclamav_rust/src directory as is canonical
for Rust projects.
Since uClibc can be configured without support for backtrace, disable
the backtrace if we are building with a uClibc that was built without
backtrace.
This is a bit hacky, and would greatly benefit from a test in ./configure
instead, but does nicely as a quick fix for now.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Bernd Kuhls <bernd.kuhls@t-online.de>
[Bernd: rebased for 0.103.0]
[Fabrice: retrieved from
https://git.buildroot.net/buildroot/tree/package/clamav/0002-mbox-do-not-use-backtrace-if-using-uClibc-without-ba.patch]
Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Add support for Rust FreeBSD, OpenBSD targets
Add support for Rust on GNU aarch64.
Add support for Rust on Alpine (musl x86_64).
Note: Current trick of checking for musl libc.so doesn't work when
cross compiling. TODO: Find a better way to check if the target is
MUSL.
Add Rust toolchain to fix Dockerfile build.
Convert cli_dbgmsg to inline function to ensure ctx check for debug flag
is always run
Add copyright and licensing info
Fix valgrind uninitialized buffer issue in cliunzip.c
Windows build fix
Vendoring crate dependencies is required for offline builds.
Some packaging systems require that builds can be performed offline.
This feature enabled vendoring crates at configure time which are
then included in in CPack source packaging.
Micah Snyder