Looking through the list of issues, I spotted some easy ones and submitted
some fixes:
- 225229 - In cli_rarload: Leak of memory or pointers to system resources.
If finding the necessary libunrar functions fails (should be rare),we now
dlclose libunrar.
225224 - In main (freshclam.c): A copied piece of code is inconsistent with
the original (CWE-398). A minor copy-paste error was present, and optOutList
could be cleaned up in one of the failure edge cases.
225228 - In decodecdb: Out-of-bounds access to a buffer (CWE-119). Off by one
error when tokenizing certain CDB sig fields for printing with sigtool. Ex:
$ cat test.cdb
a:CL_TYPE_7Z:1-2-3:/.*/:1-2-3:1-2-3:0:1-2-3::
$ cat test.cdb | ../installed/bin/sigtool --decode
VIRUS NAME: a
CONTAINER TYPE: CL_TYPE_7Z
CONTAINER SIZE: WITHIN RANGE 1 to 2
FILENAME REGEX: /.*/
COMPRESSED FILESIZE: WITHIN RANGE 1 to 2
UNCOMPRESSED FILESIZE: WITHIN RANGE 1 to 2
ENCRYPTION: NO
FILE POSITION: =================================================================
==17245==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7fffe3136d10 at pc 0x7f0f31c3f414 bp 0x7fffe3136c70 sp 0x7fffe3136c60
WRITE of size 8 at 0x7fffe3136d10 thread T0
#0 0x7f0f31c3f413 in cli_strtokenize ../../libclamav/str.c:524
#1 0x559e9797dc91 in decodecdb ../../sigtool/sigtool.c:2929
#2 0x559e9797ea66 in decodesig ../../sigtool/sigtool.c:3058
#3 0x559e9797f31e in decodesigs ../../sigtool/sigtool.c:3162
#4 0x559e97981fbc in main ../../sigtool/sigtool.c:3638
#5 0x7f0f3100fb96 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21b96)
#6 0x559e9795a1d9 in _start (/home/zelda/workspace/clamav-devel/installed/bin/sigtool+0x381d9)
Address 0x7fffe3136d10 is located in stack of thread T0 at offset 48 in frame
#0 0x559e9797d113 in decodecdb ../../sigtool/sigtool.c:2840
This frame has 1 object(s):
[32, 48) 'range' <== Memory access at offset 48 overflows this variable
HINT: this may be a false positive if your program uses some custom stack unwind mechanism or swapcontext
(longjmp and C++ exceptions *are* supported)
SUMMARY: AddressSanitizer: stack-buffer-overflow ../../libclamav/str.c:524 in cli_strtokenize
- 225223 - In cli_egg_deflate_decompress: Reads an uninitialized pointer or
its target (CWE-457). Certain fail cases would call inflateEnd on an
uninitialized stream. Now it’s only called after initialization occurs.
- 225220 - In buildcld: Use of an uninitialized variable (CWE-457). Certain
fail cases would result in oldDir being used before initialization. It now
gets zeroed before the first fail case.
- 225219 - In cli_egg_open: Leak of memory or pointers to system resources
(CWE-404). If certain realloc’s failed, several structures would not be cleaned up
- 225218 - In cli_scanhwpml: Code block is unreachable because of the syntactic
structure of the code (CWE-561). With certain macros set, there could be two
consecutive return statements.
Many of the core scanning functions' names no longer represent their
specific purpose or arguments. This commit aims to make the names more
intuitive. Names are now prefixed with "magic" if they involve
file-typing and file-type parsing. In addition, each function now
includes the type of input being scanned whether its "desc", "fmap", or
"buff". Some of the APIs also now specify "type" to indicate that a type
other than "ANY" may be passed in to select the type rather than use
file type magic for type recognition.
| current name | new name |
| ------------------------- | --------------------------------- |
| magic_scandesc() | cli_magic_scan() |
| cli_magic_scandesc_type() | <delete> |
| cli_magic_scandesc() | cli_magic_scan_desc() |
| cli_base_scandesc() | cli_magic_scan_desc_type() |
| cli_partition_scandesc() | <delete> |
| cli_map_scandesc() | magic_scan_nested_fmap_type() |
| cli_map_scan() | cli_magic_scan_nested_fmap_type() |
| cli_mem_scandesc() | cli_magic_scan_buff() |
| cli_scanbuff() | cli_scan_buff() |
| cli_scandesc() | cli_scan_desc() |
| cli_fmap_scandesc() | cli_scan_fmap() |
| cli_scanfile() | cli_magic_scan_file() |
| cli_scandir() | cli_magic_scan_dir() |
| cli_filetype2() | cli_determine_fmap_type() |
| cli_filetype() | cli_compare_ftm_file() |
| cli_partitiontype() | cli_compare_ftm_partition() |
| cli_scanraw() | scanraw() |
A way is needed to record scanned file names for two purposes:
1. File names (and extensions) must be stored in the json metadata
properties recorded when using the --gen-json clamscan option. Future
work may use this to compare file extensions with detected file types.
2. File names are useful when interpretting tmp directory output when
using the --leave-temps option.
This commit enables file name retention for later use by storing file
names in the fmap header structure, if a file name exists.
To store the names in fmaps, an optional name argument has been added to
any internal scan API's that create fmaps and every call to these APIs
has been modified to pass a file name or NULL if a file name is not
required. The zip and gpt parsers required some modification to record
file names. The NSIS and XAR parsers fail to collect file names at all
and will require future work to support file name extraction.
Also:
- Added recursive extraction to the tmp directory when the
--leave-temps option is enabled. When not enabled, the tmp directory
structure remains flat so as to prevent the likelihood of exceeding
MAX_PATH. The current tmp directory is stored in the scan context.
- Made the cli_scanfile() internal API non-static and added it to
scanners.h so it would be accessible outside of scanners.c in order to
remove code duplication within libmspack.c.
- Added function comments to scanners.h and matcher.h
- Converted a TDB-type macros and LSIG-type macros to enums for improved
type safey.
- Converted more return status variables from `int` to `cl_error_t` for
improved type safety, and corrected ooxml file typing functions so
they use `cli_file_t` exclusively rather than mixing types with
`cl_error_t`.
- Restructured the magic_scandesc() function to use goto's for error
handling and removed the early_ret_from_magicscan() macro and
magic_scandesc_cleanup() function. This makes the code easier to
read and made it easier to add the recursive tmp directory cleanup to
magic_scandesc().
- Corrected zip, egg, rar filename extraction issues.
- Removed use of extra sub-directory layer for zip, egg, and rar file
extraction. For Zip, this also involved changing the extracted
filenames to be randomly generated rather than using the "zip.###"
file name scheme.
XLM is a macro language in Excel that was used before VBA (before
1996). It is still parsed and executed by modern Excel and is gaining
popularity with malware authors.
This patch adds rudimentary support for detecting and extracting
Excel 4.0 (XLM) macros.
The code is based on Didier Steven's plugin_biff for oletools.py.
New features added to freshclam:
- Update signature definitions over HTTPS.
- Support for HTTP protocol v1.1 (formerly v1.0).
- New libfreshclam library with an all new API and versioning separate from libclamav (v2.0.0). This library is now build and installed alongside libclamav as a hard dependency of freshclam.
- The ability to opt-in and opt-out of standard and optional official ClamAV databases (ExtraDatabase, ExcludeDatabase)
- The option to specify the protocol and port number of official and private mirror servers.
- Support for additional types of proxy servers beyond plain HTTP (SOCKS 4, SOCKS 5).
Features removed from freshclam:
- Mirror management (mirrors.dat) file. This feature is no longer needed as official signature databases are distributed using a paid content delivery network (Cloudflare).
This commit also adds the following features for Windows users:
- The clamsubmit tool.
- The json-c library dependency, which will enable the --gen-json option in clamscan.
- Third party libraries under the win32/3rdparty directory have been removed. Developers will need to build the libraries separately from ClamAV and provide the headers and lib/dll library files the same way they do for OpenSSL. This includes libxml2, pthread-win32, bzip2, zlib, pcre2 as well as new dependencies: curl, json-c. Developers are encouraged to use the build tool Mussels to simplify this task.
Also, move the cert-related DCONF cfg checks to more
appropriate locations. One change in behavior:
PE_CONF_CATALOG will disable loading trusted hashes from
.cat files, but won't disable Authenticode hash checking
completely (PE_CONF_CERTS does this).
Instead of checking the Authenticode header as an FP prevention
mechanism, we now check it in the beginning if it exists. Also,
we can now do actual blacklisting with .crb rules (previously, a
blacklist rule just let you override a whitelist rule).
cli_checkfp_pe is now effectively the function that just checks
the Authenticode hash. This makes the code less complicated,
and adds some minor improvements:
- section hashes are no longer computed if there is no stats
callback function (at least in that part of the code)
- We now actually set the len field in the stats_section_t
structure
- If an error occurs when computing a section hash, we skip
that section instead of not computing any hashes
This commit adds back in support for whitelisting files based on
signatures from .cat files loaded in via a '-d' flag to clamscan.
This also makes it so that a .crb blacklist rule match can't be
overruled by a signature in a .cat file
Updated libclamav documentation detailing new scan options structure.
Renamed references to 'algorithmic' detection to 'heuristic' detection. Renaming references to 'properties' to 'collect metadata'.
Renamed references to 'scan all' to 'scan all match'.
Renamed a couple of 'Hueristic.*' signature names as 'Heuristics.*' signatures (plural) to match majority of other heuristics.
Andrew