* clamonacc: fix unused variable compile-time warning
Remove unused variable 'ret' from onas_queue.c and get rid of the
following compile-time warning:
~/clamav/clamonacc/scan/onas_queue.c: In function ‘onas_scan_queue_th’:
~/clamav/clamonacc/scan/onas_queue.c:161:9: warning: unused variable ‘ret’ [-Wunused-variable]
161 | int ret;
| ^~~
* libclamav: fix unused variable compile-time warning
Remove unused variable 'err' from libclamav/png.c, and get rid of
the following compile-time warning:
~/clamav/libclamav/png.c: In function ‘cli_parsepng’:
~/clamav/libclamav/png.c:101:9: warning: unused variable ‘err’ [-Wunused-variable]
101 | int err = Z_OK;
| ^~~
The office art structure for OLE2 documents records the file name
length using a `uint8_t`, meaning the name may be up to 255 bytes in
length, not including the null terminating byte. If the length is
255 then the parser will write the null-terminating byte just after
the end of the name buffer on the stack.
This issue does not cause a crash and is not a vulnerability.
This fix extends the size of stack array to account for the null
terminator.
Thank you Michał Dardas for reporting this issue.
The check of pattern_len against FILEBUF is largely meaningless since
pattern is derived from a strchr() call against buffer (with length FILEBUFF).
This fix ensures that the relative size is checked against max buffer size
which prevents overwriting stack memory with a single null byte.
Resolves: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=45247
The fuzz targets that write a temp file currently use the same filename
every time. One of the users identified that if the tests are running
in parallel mode, many processes are accessing the same file.
This results in unstable input to the API being tested, where the file
may be overwritten as the function is being tested.
This commit fixes it by putting the fuzz process PID in the filenames
for the scanfile and dbload fuzz targets.
Resolves: https://github.com/Cisco-Talos/clamav/issues/432
Also fixed a CMake bug that built an extra fuzz target file that doesn't
serve any purpose.
Resolves: https://github.com/Cisco-Talos/clamav/issues/431
Commit f82492aef4 fixed a crash in Windows
debug builds but in so doing accidentally introduced a possible crash
when scanning PE files that lack import tables. The issue being that
the openssl hashing functions try to "finish" a hash that was never
started.
This commit fixes the issue by returning CL_BREAK instead of CL_SUCCESS
when the import table doesn't exist or RVA is invalid so that we can
differentiate between successfully calculating the hashes and
successfully skipping the hashing process.
A heap buffer overflow could occur during resource cleanup if a
malloc fails when adding a regex pattern to the phishing suffix tree.
The solution is to increment suffix_cnt after cli_realloc succeeds.
The issue was identified using fault injection and is not a vulnerability.
Resolves: https://github.com/Cisco-Talos/clamav/issues/429
Add `sigtool --fuzzy-img` option to generate image fuzzy hash.
Also fix assorted warnings, mostly ensuring enough buffer space so format
strings aren't truncated.
For the dsig change: the returned string is allocated and is not const.
The caller will have to free it.
In 0.104 and prior, the function for adding a logical subsignature
was being used for NDB sigs, FTM, sigs, *and* Yara strings.
When cleaning up the logic for handling different types of logical
sig subsignatures, we accidentally broke support for regex strings in
yara rules.
This commit adds new logic for recording if the Yara string is a regex
string, by adding a regex subsig opt. Then in a new function for adding
different types of Yara strings, we check if it's a regex string or not
before adding as either a PCRE pattern or as an AC/BM pattern.
Resolves: https://github.com/Cisco-Talos/clamav/issues/494
Also add a basic test for yara regex rule.
Refactored the clamscan code that determines 'what to scan' in order
to clean up some very messy logic and also to get around a difference in
how vscode and clang-format handle formatting #ifdef blocks in the
middle of an else/if.
In addition to refactoring, there is a slight behavior improvement. With
this change, doing `clamscan blah -` will now scan `blah` and then also
scan `stdin`. You can even do `clamscan - blah` to now scan `stdin` and
then scan `blah`. Before, The `-` had to be the only "filename" argument
in order to scan from stdin.
In addition, added a bunch of extra empty lines or changing multi-line
function calls to single-line function calls in order to get around a
bug in clang-format with these two options do not playing nice together:
- AlignConsecutiveAssignments: true
- AlignAfterOpenBracket: true
AlignAfterOpenBracket is not taking account the spaces inserted by
AlignConsecutiveAssignments, so you end up with stuff like this:
```c
bleeblah = 1;
blah = function(arg1,
arg2,
arg3);
// ^--- these args 4-left from where they should be.
```
VSCode, meanwhile, somehow fixes this whitespace issue so code that is
correctly formatted by VSCode doesn't have this bug, meaning that:
1. The clang-format check in GH Actions fails.
2. We'd all have to stop using format-on-save in VSCode and accept the
bug if we wanted those GH Actions tests to pass.
Adding an empty line before variable assignments from multi-line
function calls evades the buggy behavior.
This commit should resolve the clang-format github action test failures,
for now.
EGG archives may have individually encrypted files, or may specify
encryption for the whole archive. For those that have individually
encrypted files, the clean-up code neglects to free the encrypt
structure, which holds 2 pointers (16 bytes on 64bit machines).
This commit adds that missing free.
Thank you Michał Dardas for reporting this issue.
Also add release notes from 0.103 and 0.104 patch versions published
during the development of 0.105.
Also Update llvm support details in the INSTALL.md file.
CMake has a pretty sweet alternative to pkg-config or to the older
`package-config` script that you'd run with `--libs`, `--ldflags`, etc.
CMake can (optionally) install a YourPackageConfig.cmake alongside your
libs under `<prefix>/lib/cmake/<pkg>/<pkg>Config.cmake`.
If you build something with -D CMAKE_FIND_PACKAGE_PREFER_CONFIG=TRUE,
then it will find that and use that to bring real CMake targets into
your build system for your dependencies, guaranteeing that you get all
the right include paths, library paths, ldflags, etc.
See: https://cmake.org/cmake/help/latest/variable/CMAKE_FIND_PACKAGE_PREFER_CONFIG.html
This works great for us for Curl and LLVM... not so much for libxml2.
Ideally we'd figure out what's wrong and support libxml2's package
config.cmake file, but for now this workaround lets us ignore libxml2
and continu to use this feature for the other libs, like LLVM.
I observed undefined symbol errors when linking the bytecode_runtime
object with the check_* test executables on Windows when using LLVM
built from source. I'm not sure why, exactly, but these symbols should
all be present in the ClamAV::libclamav library that we're linking with,
so I don't know why we link with the object library targets in addition
to the libclamav shared/or/static library target.
This commit removes linking with those extra object library targets.
The `have_clamjit` global is used in the unit tests but doesn't appear
to be exported when I was testing the external LLVM runtime support PR,
resulting in an undefined symbol issue. Converting this to a function
that returns 0 or 1 instead of a global variable resolved the issue.
We must pass the LLVM library dependencies to the libclamav_rust
build.rs script so it links the libclamav_rust unit test executable with
LLVM.
Also:
- We can remove the libtinfo dependency that was hardcoded for the LLVM
3.6 support, and must remove it for the build to work on Alpine, macOS.
- Also, increased the libcheck default timeout from 60s to 300s after
experiencing a failure while testing this.
- Also made one of the valgrind suppressions more generic to account for
inline optimization differences observed in testing this.
Modified bytecode JIT code to support llvm versions 8, 9, 10, 11, 12.
Additionally updated FindLLVM.cmake to the current version, found at
https://github.com/ldc-developers/ldc/blob/master/cmake/Modules/FindLLVM.cmake,
as well as making modifications suggested by Micah Snyder to check VERSION_LESS
in CMakeLists.txt to avoid having modifications to FindLLVM.cmake.
Validate the length of the crt subject before memcpying it.
This resolves a possible multibyte heap buffer overread.
We determiend that this issue is not a vulnerability.
Fixes: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43828
The endianness conversion when reading the PE image import descriptor is
making the change in-place in the fmap.
On Windows, the fmap is read-only and so in Debug builds that's causing
a crash.
This uses a buffer on the stack and copies each image import descriptor
before doing the conversions and then processing each.
* Broke out the variants of error/result handling in `frs_error.rs`.
Made syntax slightly cleaner for `frs_call!`, explicitly moving
the output variables *out* of the function call so as not to make
the parameter order confusing.
* Wrapped the FuzzyHash map into a container rather than exposing
the HashMap directly. Simplifies casting, and allows it to feel
more like a class with methods.
* Fixed various clippy complaints regarding unsafe, etc.
* Rename `frs_error.rs` to `ffi_utils.rs` and migrated ffi-specific
features like the `validate_str_param!()` macro to this new module.
Extends the new frs_error module to provide variants of the
frs_result!() macro that accept a Result as input instead of calling a
function on your behalf. This enables us to use the macro in conditions
where we don't want to return on success, and want to do other things
before we return.
Use the new frs_error module to return errors to the C calling functions
rather than logging the error in Rust-land.
Notably, this enables us to store more meaningful error messages in the
JSON output if we fail to calculate the image fuzzy hash.
The new frs_error module makes it so you can painlessly transfer Rust
errors to-and-from C.
On the Rust-side, on failure it returns an error struct to the C caller.
The C caller detects the failure by checking the return value.
On failure, the caller may use an additional frs_error Rust API to get a
formatted error message.
Two variants are provided:
- One for functions that return a boolean and have an out-parameter for
function output.
- One for functions that return a pointer, where NULL means failure and
non-NULL contains the successful function output.
We've observed a number of false positives with iconv, pthread, libxml2,
etc regarding memory allocation on init that is never cleaned up but
isn't a leak. Now we're seeing a whole tonne more from Rust libraries
with wild stack traces that couldn't be easily suppress.
So the solution is to disable any alerts from possible-leaks and only
show definite-leaks.
Updates to image fuzzy hash algorithm to mimick the phash() function
from the Python imagehash package as closely as possible.
The primary difference is that the DCT is now a 2D DCT-2 instead of
a standard DCT-2 on a flattened array of the image data.
We also needed to:
- drop the alpha channel for images with transparency.
- use the grayscale coefficients used by Python's Pillow library.
- round rather than truncate when converting from f32 -> u8.
- use a median instead of a mean for finding the average frequency in
the low-frequency vector. Previously I had though we were
implementing phash_simple(), which used mean. Sorry Mickey!
- multiply the Rust DCT-2 results * 2, to match DCT results from the
Python package.
Note:
The image-rs crate (v0.24) doesn't currently support using custom
RGB->Luma constants and doesn't round when converting Luma f32 back to
Luma u8 values.
So, this commit includes a custom greyscale() function just for rgb8
images so we can match the greyscale algorithm used in the Python
Pillow package.
See also: https://github.com/image-rs/image/issues/1554
The change in the code that converts the hash bit-vector to a Vec<u8>,
removing map_while(), was done to support older Rust versions,
such as the one currently in Alpine 3.15.0.
Add a new logical signature subsignature type for matching on images
with image fuzzy hashes.
Image fuzzy hash subsigantures follow this format:
fuzzy_img#<hash>#<dist>
In this initial implementation, the hamming distance (dist) is ignored
and only exact fuzzy hash matches will alert.
Fuzzy hash matching is only performed for supported image types.
Also: removed some excessive debug log messages on start-up.
Fixed an issue where the signature name (virname) is being allocated and
stored for every subsignature or even ever sub-pattern in an AC-pattern
(i.e. NDB sig or LDB subsig) containing a `{n-m}` or `*` wildcard.
This fix is only for LDB subsigs though. NDB signatures are still
allocaing one virname per sub-pattern.
This fix was required because I needed a place to store the virname with
fuzzy-hash subsignatures. Storing it in the fuzzy-hash subsig
metadatathe way AC-pattern, PCRE, and BComp subsigs were doing it
wouldn't work because it would cross the C-Rust FFI boundary and giving
pointers to Rust allocated stuff is dicey. Not to mention native Rust
strings are different thatn C strings. Anyways, the correct thing to do
was to store the virname with the actual logical signature.
TODO: Keep track of NDB signatures in the same way and store the virname
for NDB sigs there instead of in AC-patterns so that we can get rid of
the virname field in the AC-pattern struct.
Fix two locations where the stack-allocated arrays lack space for a null-
terminating byte and could overwrite the array in:
- dsig.c
- sigtool.c
The ClamAV team verified that these overflows are not a security issue.
The current method of trying to determine the target triple based
the architecture, operating system, etc. is difficult to get right
and maintain. In particular, I found that some installations like the
Alpine package will use "alpine" in the name of the triple instead of
"unknown".
E.g. "aarch64-alpine-linux-musl" instead of "aarch64-unknown-linux-musl".
This makes it nearly impossible to figure out the exact target name
based on target system specifications.
I believe it will be better to use the default target 99% of the time
and require users that which to cross-compile to use a new CMake
variable `-D RUST_TARGET_TRIPLE=<target>` to choose the target.
This is basically how it works for C/C++ anyways.
The clamonacc-ddd thread (clamonacc/inotif/inotif.c) has currently no
handling for inotify events where the passed watch descriptor is lower
than zero (e.g.: "-1"). Although not actually mentioned in the inotify
documentation, this case has actually been observed:
[...]
ClamInotif: inotify event wd:-1, mask:1073742080, cookie:0, length:16, child:<CHILD-PATH>
Clamonacc: onas_clamonacc_exit(), signal 11
Assumption: This probably occurs when a inotify event is generated for
the creation of a directory including subdirectories. The clamonacc-ddd
thread then discovers all children of the directory and is generating
inotify watch descriptors for all children. A subsequent inotify event
is generated for a subdirectory, but an inotify watch descriptor already
exists from the previous discovery.
In any case, this leads to an out-of-bounds array access on the internal
data structure "wdlt" of clamonacc while trying to look up the "path"
from there. This in turn can lead to a SIGSEGV of the clamonacc-ddd
thread.
The clamonacc-ddd thread (clamonacc/inotif/inotif.c) has currently no
explicit handling for the special inotify kernel events IN_UNMOUNT,
IN_Q_OVERFLOW and IN_IGNORED. It treats the inotify_event structs in
the same way as regular inotify events, although for those special
events the "len" and "name" fields a value of zero respectively NULL.
This can lead to a SIGSEGV of the clamonacc-ddd thread when calling
"strlen(name)" on a NULL value.
This commit adds just a quick'n'dirty workaround for the SIGSEGV and
some logging output. It DOES NOT fix the more overall issue, that a
IN_Q_OVERFLOW inotify event also leads to an out-of-sync situation
between the monitored filesystem/directory and the internal data
structures "ddd_ht" and "wdlt" of clamonacc.
The clamonacc-ddd thread (clamonacc/inotif/inotif.c) currently and in-
discriminately stops processing of inotify watch descriptors in the
function "onas_ddd_unwatch_hierarchy" upon any error returned from the
call to "inotify_rm_watch". This in turn also prevents further updates
and cleanup of the internal data structures "ddd_ht" and "wdlt", which
leads to a invalid value of the "path" variable.
This causes a leak of inotify watch descriptors in the case of e.g. a
move of a directory containing a hierarchy of watched subdirectories.
See the inotify watch descriptor ("wd:") values in the section "Example
and reproducer" below.
This commit adds a workaround for this issue by specifically ignoring
the "ENOENT" errno in case the call to "inotify_rm_watch" results in
a non-zero return code. This allows the cleanup to properly proceed
for all children.
Example and reproducer:
- Starting point:
Log:
ClamInotif: created watch descriptor (wd:1) for path: /clamonacc/monitored
[...]
ClamInotif: created watch descriptor (wd:129637) for path: /clamonacc/monitored/[...]
- Initial directory and subdirectory creation:
mkdir -p /clamonacc/monitored/.test_ff/{1,2,3,4,5}
Log:
ClamInotif: inotify event wd:1, mask:1073742080, cookie:0, length:16, child:.test_ff
ClamInotif: inotify event path: /clamonacc/monitored
ClamInotif: CREATE - adding /clamonacc/monitored/.test_ff to /clamonacc/monitored with wd:1
ClamInotif: created watch descriptor (wd:129638) for path: /clamonacc/monitored/.test_ff
ClamInotif: created watch descriptor (wd:129639) for path: /clamonacc/monitored/.test_ff/3
ClamInotif: created watch descriptor (wd:129640) for path: /clamonacc/monitored/.test_ff/4
ClamInotif: created watch descriptor (wd:129641) for path: /clamonacc/monitored/.test_ff/1
ClamInotif: created watch descriptor (wd:129642) for path: /clamonacc/monitored/.test_ff/5
ClamInotif: created watch descriptor (wd:129643) for path: /clamonacc/monitored/.test_ff/2
- Move / rename of the parent directory:
mv -i /clamonacc/monitored/.test_ff/ /clamonacc/monitored/test_ff
Log:
ClamInotif: inotify event wd:1, mask:1073741888, cookie:12094045, length:16, child:.test_ff
ClamInotif: inotify event path: /clamonacc/monitored
ClamInotif: MOVED_FROM - removing /clamonacc/monitored/.test_ff from /clamonacc/monitored with wd:1
ERROR: ClamInotif: error removing watch descriptor (wd:129638) for path: /clamonacc/monitored/.test_ff, No such file or directory
ClamInotif: inotify event wd:1, mask:1073741952, cookie:12094045, length:16, child:test_ff
ClamInotif: inotify event path: /clamonacc/monitored
ClamInotif: MOVED_TO - adding /clamonacc/monitored/test_ff to /clamonacc/monitored with wd:1
ClamInotif: created watch descriptor (wd:129644) for path: /clamonacc/monitored/test_ff
ClamInotif: created watch descriptor (wd:129639) for path: /clamonacc/monitored/test_ff/3
ClamInotif: created watch descriptor (wd:129640) for path: /clamonacc/monitored/test_ff/4
ClamInotif: created watch descriptor (wd:129641) for path: /clamonacc/monitored/test_ff/1
ClamInotif: created watch descriptor (wd:129642) for path: /clamonacc/monitored/test_ff/5
ClamInotif: created watch descriptor (wd:129643) for path: /clamonacc/monitored/test_ff/2
- Removal of the parent directory unter the new name:
rm -r /clamonacc/monitored/test_ff
Log:
ClamInotif: inotify event wd:129644, mask:1073742336, cookie:0, length:16, child:3
ClamInotif: inotify event path: /clamonacc/monitored/test_ff
ClamInotif: DELETE - removing /clamonacc/monitored/test_ff/3 from /clamonacc/monitored/test_ff with wd:129644
ClamInotif: removed watch descriptor (wd:129639) for path: /clamonacc/monitored/test_ff/3
ClamInotif: inotify event wd:129644, mask:1073742336, cookie:0, length:16, child:4
ClamInotif: inotify event path: /clamonacc/monitored/test_ff
ClamInotif: DELETE - removing /clamonacc/monitored/test_ff/4 from /clamonacc/monitored/test_ff with wd:129644
ClamInotif: removed watch descriptor (wd:129640) for path: /clamonacc/monitored/test_ff/4
ClamInotif: inotify event wd:129644, mask:1073742336, cookie:0, length:16, child:1
ClamInotif: inotify event path: /clamonacc/monitored/test_ff
ClamInotif: DELETE - removing /clamonacc/monitored/test_ff/1 from /clamonacc/monitored/test_ff with wd:129644
ClamInotif: removed watch descriptor (wd:129641) for path: /clamonacc/monitored/test_ff/1
ClamInotif: inotify event wd:129644, mask:1073742336, cookie:0, length:16, child:5
ClamInotif: inotify event path: /clamonacc/monitored/test_ff
ClamInotif: DELETE - removing /clamonacc/monitored/test_ff/5 from /clamonacc/monitored/test_ff with wd:129644
ClamInotif: removed watch descriptor (wd:129642) for path: /clamonacc/monitored/test_ff/5
ClamInotif: inotify event wd:129644, mask:1073742336, cookie:0, length:16, child:2
ClamInotif: inotify event path: /clamonacc/monitored/test_ff
ClamInotif: DELETE - removing /clamonacc/monitored/test_ff/2 from /clamonacc/monitored/test_ff with wd:129644
ClamInotif: removed watch descriptor (wd:129643) for path: /clamonacc/monitored/test_ff/2
ClamInotif: inotify event wd:1, mask:1073742336, cookie:0, length:16, child:test_ff
ClamInotif: inotify event path: /clamonacc/monitored
ClamInotif: DELETE - removing /clamonacc/monitored/test_ff from /clamonacc/monitored with wd:1
ClamInotif: removed watch descriptor (wd:129644) for path: /clamonacc/monitored/test_ff
Răzvan Cojocaru