This patch adds experimental-quality CMake build tooling.
The libmspack build required a modification to use "" instead of <> for
header #includes. This will hopefully be included in the libmspack
upstream project when adding CMake build tooling to libmspack.
Removed use of libltdl when using CMake.
Flex & Bison are now required to build.
If -DMAINTAINER_MODE, then GPERF is also required, though it currently
doesn't actually do anything. TODO!
I found that the autotools build system was generating the lexer output
but not actually compiling it, instead using previously generated (and
manually renamed) lexer c source. As a consequence, changes to the .l
and .y files weren't making it into the build. To resolve this, I
removed generated flex/bison files and fixed the tooling to use the
freshly generated files. Flex and bison are now required build tools.
On Windows, this adds a dependency on the winflexbison package,
which can be obtained using Chocolatey or may be manually installed.
CMake tooling only has partial support for building with external LLVM
library, and no support for the internal LLVM (to be removed in the
future). I.e. The CMake build currently only supports the bytecode
interpreter.
Many files used include paths relative to the top source directory or
relative to the current project, rather than relative to each build
target. Modern CMake support requires including internal dependency
headers the same way you would external dependency headers (albeit
with "" instead of <>). This meant correcting all header includes to
be relative to the build targets and not relative to the workspace.
For example, ...
```c
include "../libclamav/clamav.h"
include "clamd/clamd_others.h"
```
... becomes:
```c
// libclamav
include "clamav.h"
// clamd
include "clamd_others.h"
```
Fixes header name conflicts by renaming a few of the files.
Converted the "shared" code into a static library, which depends on
libclamav. The ironically named "shared" static library provides
features common to the ClamAV apps which are not required in
libclamav itself and are not intended for use by downstream projects.
This change was required for correct modern CMake practices but was
also required to use the automake "subdir-objects" option.
This eliminates warnings when running autoreconf which, in the next
version of autoconf & automake are likely to break the build.
libclamav used to build in multiple stages where an earlier stage is
a static library containing utils required by the "shared" code.
Linking clamdscan and clamdtop with this libclamav utils static lib
allowed these two apps to function without libclamav. While this is
nice in theory, the practical gains are minimal and it complicates
the build system. As such, the autotools and CMake tooling was
simplified for improved maintainability and this feature was thrown
out. clamdtop and clamdscan now require libclamav to function.
Removed the nopthreads version of the autotools
libclamav_internal_utils static library and added pthread linking to
a couple apps that may have issues building on some platforms without
it, with the intention of removing needless complexity from the
source. Kept the regular version of libclamav_internal_utils.la
though it is no longer used anywhere but in libclamav.
Added an experimental doxygen build option which attempts to build
clamav.h and libfreshclam doxygen html docs.
The CMake build tooling also may build the example program(s), which
isn't a feature in the Autotools build system.
Changed C standard to C90+ due to inline linking issues with socket.h
when linking libfreshclam.so on Linux.
Generate common.rc for win32.
Fix tabs/spaces in shared Makefile.am, and remove vestigial ifndef
from misc.c.
Add CMake files to the automake dist, so users can try the new
CMake tooling w/out having to build from a git clone.
clamonacc changes:
- Renamed FANOTIFY macro to HAVE_SYS_FANOTIFY_H to better match other
similar macros.
- Added a new clamav-clamonacc.service systemd unit file, based on
the work of ChadDevOps & Aaron Brighton.
- Added missing clamonacc man page.
Updates to clamdscan man page, add missing options.
Remove vestigial CL_NOLIBCLAMAV definitions (all apps now use
libclamav).
Rename Windows mspack.dll to libmspack.dll so all ClamAV-built
libraries have the lib-prefix with Visual Studio as with CMake.
Removed all autotools generates files. Autotools (autoconf, automake,
libtool, pkg-config, m4) will be required from now on for builds from
git clones.
Added autogen.sh to be run before ./configure.
Significant update to main .gitignore file.
Removed extraneous .gitignore files. A Git repository only needs one
.gitignore file.
Fixed the following Coverity issues:
- 225236 - In cli_egg_extract_file: Dereference of an explicit
null value (CWE-476). The first fail case checked handle for
NULL and then dereferenced it in the done block
- 225209 - In executeIfNewVersion: Leak of memory or pointers
to system resources (CWE-404). modifiedCommand was defined
twice, with the inner instance being assigned to and the
outer instance being freed
- 225201 - In regex_list_match: Code can never be reached
because of a logical contradiction (CWE-561). The code had
logic off to the side that may have been missed:
filter_search_rc = filter_search(&matcher->filter, (const unsigned char *)bufrev, buffer_len) != -1;
if (filter_search_rc == -1) {
- 225198 - In phishingCheck: Leak of memory or pointers to
system resources (CWE-404). A fail case caused by malloc
failing would leak previously allocated memory.
- 225197 - In updatecustomdb: A pointer to freed memory
is dereferenced, used as a function argument, or otherwise
used (CWE-416). In a fail case, a pointer was freed and
then used in a debug print statement
- 225190 - In updatedb: A pointer to freed memory is
dereferenced, used as a function argument, or otherwise used
(CWE-416). In a fail case, a pointer was freed and then used
in a debug print statement
- 225195 - In cli_egg_open: The sizeof operator is used on a
wrong argument that incidentally has the same size (CWE-467).
sizeof(char **) was being used instead of sizeof(char *)
- 225193 - In egg_parse_comment_header: Code can never be
reached because of a logical contradiction (CWE-561).
A cleanup case for variable comment was unnecessary, and
to fix comment was removed entirely.
- 225147 - In get_server_node: Code can never be reached
because of a logical contradiction (CWE-561). A cleanup
case for variable url was unnecessary
- 225168 - In download_complete_callback: Missing break
statement between cases in switch statement (CWE-484).
In the case where forking failed, freshclam would check
the database without forking but then continue on to
execute the code intended to be done in the child process
because of a missing break statement
- 225152 - In cli_egg_lzma_decompress: Use of an
uninitialized variable (CWE-457). Certain fail cases
would call cli_LzmaShutdown on an uninitialized stream.
Now it’s only called after initialization occurs.
Looking through the list of issues, I spotted some easy ones and submitted
some fixes:
- 225229 - In cli_rarload: Leak of memory or pointers to system resources.
If finding the necessary libunrar functions fails (should be rare),we now
dlclose libunrar.
225224 - In main (freshclam.c): A copied piece of code is inconsistent with
the original (CWE-398). A minor copy-paste error was present, and optOutList
could be cleaned up in one of the failure edge cases.
225228 - In decodecdb: Out-of-bounds access to a buffer (CWE-119). Off by one
error when tokenizing certain CDB sig fields for printing with sigtool. Ex:
$ cat test.cdb
a:CL_TYPE_7Z:1-2-3:/.*/:1-2-3:1-2-3:0:1-2-3::
$ cat test.cdb | ../installed/bin/sigtool --decode
VIRUS NAME: a
CONTAINER TYPE: CL_TYPE_7Z
CONTAINER SIZE: WITHIN RANGE 1 to 2
FILENAME REGEX: /.*/
COMPRESSED FILESIZE: WITHIN RANGE 1 to 2
UNCOMPRESSED FILESIZE: WITHIN RANGE 1 to 2
ENCRYPTION: NO
FILE POSITION: =================================================================
==17245==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7fffe3136d10 at pc 0x7f0f31c3f414 bp 0x7fffe3136c70 sp 0x7fffe3136c60
WRITE of size 8 at 0x7fffe3136d10 thread T0
#0 0x7f0f31c3f413 in cli_strtokenize ../../libclamav/str.c:524
#1 0x559e9797dc91 in decodecdb ../../sigtool/sigtool.c:2929
#2 0x559e9797ea66 in decodesig ../../sigtool/sigtool.c:3058
#3 0x559e9797f31e in decodesigs ../../sigtool/sigtool.c:3162
#4 0x559e97981fbc in main ../../sigtool/sigtool.c:3638
#5 0x7f0f3100fb96 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21b96)
#6 0x559e9795a1d9 in _start (/home/zelda/workspace/clamav-devel/installed/bin/sigtool+0x381d9)
Address 0x7fffe3136d10 is located in stack of thread T0 at offset 48 in frame
#0 0x559e9797d113 in decodecdb ../../sigtool/sigtool.c:2840
This frame has 1 object(s):
[32, 48) 'range' <== Memory access at offset 48 overflows this variable
HINT: this may be a false positive if your program uses some custom stack unwind mechanism or swapcontext
(longjmp and C++ exceptions *are* supported)
SUMMARY: AddressSanitizer: stack-buffer-overflow ../../libclamav/str.c:524 in cli_strtokenize
- 225223 - In cli_egg_deflate_decompress: Reads an uninitialized pointer or
its target (CWE-457). Certain fail cases would call inflateEnd on an
uninitialized stream. Now it’s only called after initialization occurs.
- 225220 - In buildcld: Use of an uninitialized variable (CWE-457). Certain
fail cases would result in oldDir being used before initialization. It now
gets zeroed before the first fail case.
- 225219 - In cli_egg_open: Leak of memory or pointers to system resources
(CWE-404). If certain realloc’s failed, several structures would not be cleaned up
- 225218 - In cli_scanhwpml: Code block is unreachable because of the syntactic
structure of the code (CWE-561). With certain macros set, there could be two
consecutive return statements.
Currently, freshclam's "OnUpdateExecute", "OnErrorExecute", and
"OnOutdatedExecute" options will fail when running on Windows in daemon
mode if the supplied command contains any arguments.
This patch fixes execute() function on Windows in daemon mode by
replacing spawnlp() with system().
The commit "Freshclam create database directory if missing" inadvertently
broke the build, because it was blindly rebased and merged after a prior
commit relocated the required `statbuf` variable.
This commit adds back the missing `statbuf` variable.
On initialization, freshclam will create the database directory if it is
missing.
If running as root, freshclam will assign ownership of the new directory
to the DatabaseOwner account.
Freshclam creates a tmp directory in the database directory used to
store downloaded patches or databases before they replace current
databases. The tmp directory previously was created at when freshclam
was initialized and deleted when freshclam exited. This was problematic
if freshclam was run in daemon mode and then run manually while the
daemon was already running.
This commit alters the behavior to create tmp directory with a random
suffix before the update begins and remove this directory when the
update ends, allowing freshclam to be run manually without causing the
freshclam daemon to fail later.
New features added to freshclam:
- Update signature definitions over HTTPS.
- Support for HTTP protocol v1.1 (formerly v1.0).
- New libfreshclam library with an all new API and versioning separate from libclamav (v2.0.0). This library is now build and installed alongside libclamav as a hard dependency of freshclam.
- The ability to opt-in and opt-out of standard and optional official ClamAV databases (ExtraDatabase, ExcludeDatabase)
- The option to specify the protocol and port number of official and private mirror servers.
- Support for additional types of proxy servers beyond plain HTTP (SOCKS 4, SOCKS 5).
Features removed from freshclam:
- Mirror management (mirrors.dat) file. This feature is no longer needed as official signature databases are distributed using a paid content delivery network (Cloudflare).
This commit also adds the following features for Windows users:
- The clamsubmit tool.
- The json-c library dependency, which will enable the --gen-json option in clamscan.
- Third party libraries under the win32/3rdparty directory have been removed. Developers will need to build the libraries separately from ClamAV and provide the headers and lib/dll library files the same way they do for OpenSSL. This includes libxml2, pthread-win32, bzip2, zlib, pcre2 as well as new dependencies: curl, json-c. Developers are encouraged to use the build tool Mussels to simplify this task.
Micah Snyder (micasnyd)