clamscan_test.py was accidentally re-introduced with commit
35ccb49f38
All of the tests in this file are covered in other _test.py files
in the unit_tests/clamscan directory.
The bytecode compiler has a bug that was observed to cause a crash
with a specific signature when testing on FreeBSD.
This PR adds an index size check when looking up a constant to
prevent this crash in case a bugged signature gets through QA.
An additional NULL check is added just to be safe.
The CVD/CLD unpack calls performed by sigtool didn't used to verify the
CVD. If working with a CLD, verifying will fail, such as when using
`sigtool --find-sig` when daily is a CLD.
This commit reverts that behavior for sigtool unpack operations.
A race condition existed where clamonacc would call logg and attempt to
write to a logfile either before the logg interface had been initialized
or after it had been cleaned up.
This happens due to logg calls at cleanup during asynchronous thread
shutdowns, and during startup when watching directories with ongoing
event triggers.
This resulted in new files with garbage-filled names being created and
written to under the initial process' runtime path.
Changes:
Move logg setup to start of clamonacc.c main()
Change all raceable calls to logg to mprintf instead
If the image decoders (i.e. jpeg, tiff, png, etc.) fail to load an image
due to a panic, the application will crash.
This commit attempts to catch those panics and handle the error.
While reviewing scan time performance, the time limit checks have a
notable effect on scan time. I tested reducing the number of time checks
to every n'th signature evaluation. For the given sample which has a
large number of very tiny embedded scans, I found that overall scan time
was best when checked every 10th signature. Reducing the number of
checks any further had no noticable effect.
Bear in mind that these numbers include daily/main/bytecode CVD load
time:
| current | mod 2 | mod 10 |
| ----------------- | ----------------- | ----------------- |
| 63.848 s ±1.188 s | 61.773 s ±0.652 s | 59.831 s ±0.975 s |
| mod 50 | mod 100 | mod 1000 |
| ----------------- | ----------------- | ----------------- |
| 59.279 s ±1.652 s | 59.198 s ±1.147 s | 59.440 s ±1.304 s |
ClamAV requires non-default build options for TomsFastMath to support
bigger floating point numbers. Without this change, database and
Windows EXE/DLL authenticode certificate validation may fail.
The option to build with an external TomsFastMath library should be
removed. The `ENABLE_EXTERNAL_TOMSFASTMATH` build should be ignored,
with some sort of warning.
This commit removes the option. If used, the build will not fail, but
CMake will print a warning that the unused option was ignored.
This bug was introduced during 1.0 development and does not affect any
stable release.
An OLE2 document that claims to have a sector size (big block size) over
28 will cause a signed integer overflow for our local off_t variable,
when multiplied by 4. This will then result in an invalid read and
segfault.
The fix is two-fold:
1. Make sure the sector size (big block size) is not greater
than 28, so we don't accidentaly overflow an int32_t.
2. Perform the size check _before_ using the variable.
This commit also fixes some variable types from `unsigned int` or
`off_t` over to the more appropriate `size_t`, and from `int` or
`uint32_t`, over to `bool`.
Fixes: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=52639
When adding the `cl_cvdunpack()` API that (optionally) verifies the
database signature, we used it in libfreshclam in a place where it may
also unpack CLD database archives. CLD's may not be verified, because
the signature information is no longer valid after incremental update.
This commit fixes the issue by only verifying the unpack if the file is
a CVD and not a CLD.
If you run clamdscan with the --allmatch option, it will cause all
subsequent clamdscan scans to have all-match mode enabled.
This bug is specific to clamd / clamdscan and does not affect clamscan.
The problem was introduced when we converted the scan options from a
single integer bitfield to a struct. The scan options set by the
clamdscan parameters should be saved in a local copy of the scan
options, but instead it is saving a copy of the pointer to the scan
options struct, and so any changes to the scan options affect future
scans.
In preparation for the release candidate, changing the version suffix
from devel to rc.
Increment SO current version to 11, because we added to the public API.
Increment SO age to 2, because we didn't change anything or remove
anything from the public API.
Added functionality level values to the bytecode API enum for recent
patch versions.
Coverity found two slightly different buffer length issues
with the memcmp at the end of the verify_key_aes() function.
The problem is that the hash must be a SHA1, but the the verifier hash
field is up to 32 bytes, and the structure may also lie about the size
of the hash, saying it's more than 32 bytes.
Realistically, it should be 20 bytes if it is functioning correctly,
so we shouldn't really care about the extra 12 bytes.
When LLVM is found with FindLLVM.cmake module, it may set the
LLVM_LIBRARIES variable to have `-l` prefix on the libraries.
The Rust `build.rs` script does not like this and will end up trying to
link with `-l-lLLVM` instead of `-lLLVM`.
This commit strips the `-l` prefix from the LLVM_LIBRARIES variable
before passing it off to build the libclamav_rust test program.
Coverity static analysis is complaining that the ctx variable
is checked for NULL after it has already been dereferenced.
This code was copied from elsewhere in the file, so fixed it
there as well.
Add error checking to make sure we aren't calling into aes decrypt
functions with larger key sizes, that would make decrypt read past the
end of it's data array.
CMake's configuration step fails if you use ENABLE_LIBCLAMAV_ONLY=ON and
ENABLE_TESTS=ON. This is because we try to add the libcurl module path
to the test environment, for the sake of linking with the Rust unit test
program, as we do with the other dependencies. However, libcurl is not
required or even found in libclamav-only mode. So that fails.
The PE cert test can be enabled now that the cert trust feature is
fixed. In so doing I found an issue with it -- it was also using the
block-certificate signature, which overrides the trust-certificate
signatures. This made me realize that we should also have a test to make
sure the block-cert signatures take predence over the trust-cert sigs.
I fixed the original sig and added this second test case.
The function `cli_scan_fmap()` is where we check PE authenticode
certificates. But it is only returning CL_CLEAN, CL_VIRUS, or file
types. It should propagate errors as well as CL_VERIFIED, if the
authenticode finds we should trust the file.
RSA certificate verification is failing because we accidentally lowered
the max size of numbers for floating point multiplication with
TomsFastMath when upgrading the vendored library code.
This commit restores the default from 4096 to 8192.
Adds in object extraction recursion limits for object extraction as well
as well parsing string, array, and dictionaries during extraction.
The limit is set to 25.
Places recursion-depth variable in pdf parse context structure.
Signatures that have conditions on a Container type or Intermediates
types may not match if the target file is scanned outside of the
container first, and cached as clean.
The solution to this is to disable caching for layers that would
have matched except for the Container (or Intermediates) condition.
Increases internal max allocation size to allow support for
files up to 1GB.
Provides a more helpful, less dramatic warning message if this
limit is exceeded.
Some small files that appear to include archive content (zip entries,
RAR archives, etc) later in the file are no longer set to that SFX type
initially. However, the current way we skip those drops the initial
type detection on the floor, meaning that something like a small image
may end up as CL_TYPE_ANY instead of CL_TYPE_PNG.
This commit fixes that by changing where in the process the SFX types
are ignored.
Some files that take longer to scan may exceed the scan time by a fair
bit during the actual pattern matching part of the scan (vs.
decompressing archives or running bytecode functions, or whatever).
ClamAV does pattern matching and hashing of files in chunks.
This change adds a time check before each chunk is matched.
In the logical sig evaluation function, we are calling the run bytecode
function even if the lsig doesn't depend on bytecode. That function
immediately returns CL_ENULLARG, which previously was ignored because we
were only checking for CL_VIRUS. Now that we're checking if NOT success,
the error propagates and scanning of that layer ends early.
The above bug is actually in some duplicate code, so when fixing this I
went ahead and refactored the function to remove this, and added
additional comments and readability improvements.
Error handling for bytecode sig evaluation, as well as logical and yara
sig evaluation only consider CL_VIRUS or CL_SUCCESS and fail to
consider terminal errors such as CL_ETIMEOUT.
This commit fixes the error handling in those functions so we properly
abort instead of continuing to evaluate more sigs.
This commit also adds some a scan time limit checks:
1. shortly after a bytecode time limit exceeded.
2. after evaluating each lsig or yara sig.
3. when processing unzip central directory file headers.
The `cli_memcpy` function is a version of memcpy that will catch
exceptions on Windows if the memcpy fails because the memory-mapped file
fails to read (like if a network drive is disconnected or something).
It presently returns an `int`. Since it is only used in one spot, I've
swapped it over to a `cl_error_t`, and created a unique variable so
we're not piggybacking with the `cli_file_t` enum used to check the
result of a filetyping scan.
Fix pretty silly double-free, and fix issue where 'extracted_file' was
not being set to true.
Also rename `uncompressed` variable to `uncompressed_block`, because
there is another variable with the same name, above.
Because of an oversight, the `ctx->abort_scan` flag was propagating
timeout and perhaps other error types to the final verdict.
Fixed this in the `result_should_goto_done()` function.
Some non-fatal errors (notably CL_ETIMEOUT) may appear between the call
to the magic-scan, and the end of scan_common().
We need to filter these status results at this level as well to prevent
this from happening.
I also fixed a warning accidentally introduced earlier caused by trying
to use a string after free'ing it and setting it to NULL.
And I explicitly set the status to CL_SUCCESS if the cache check comes
back clean, rather than trusting that no one had messed with the `ret`
variable since it was let set (to CL_SUCCESS). It's more readable and
less prone to future bugs.
Micah Snyder