To fix an old (and probably no longer manifest) build bug, this commit
adds the ability to detect an 8-bit libpcre2 with pkg-config in
addition to the other methods of specifying/detecting it. The
intention is that pkg-config will be used only in the default case,
where pcre support is detected automatically; that is, NOT in either
of these cases:
* --with-pcre=<path> is given
* --with-pcre=no is given
The code in pcre.m4 was modified minimally so that --with-pcre=yes
(the default) tries pkg-config first, and then falls back to whatever
it used to do. If pkg-config can find the library, we add PCRE_LIBS
to the LIBS variable and we're done. Otherwise, the old behavior is
retained.
ClamAV-bug: https://bugzilla.clamav.net/show_bug.cgi?id=12484
Gentoo-bug: https://bugs.gentoo.org/567680
This commit also collects the preprocessor flags obtained from
pkg-config.
When libpcre2 is found using pkg-config, we now say that explicitly,
and include its PCRE_LIBS and PCRE_CFLAGS in the output.
clamscan.exe fails to delete a file from C:\somefolder though it can
successfully delete a file from a user-owned directory (eg
C:\Users\user\somefolder). From what I can tell, this is because a new
directory in C:\somefolder does not have the full-access ACL by
default.
This patch changes the NtCreateFile() call to open with generic read
& delete rights, instead of requesting full-access.
The pcre2.h header dependency is propagated to the bytecode runtime,
lzma_sdk, yara, and regex build targets within the libclamav build
because it is included by matcher.h which is included all over the
place.
This patch adds the pcre2 dependency to the affected build targets so
that systems where pcre2 isn't in the standard include path can still
build.
Also removed CMake `PCRE2_DIR` from documentation, as it doesn't apply
to this PCRE2 detection logic that we settled on.
There is an autotools variable and clamav-config.h macro BUILD_CLAMD
which dates back 17 years to a time where libpthread support was
optional and users could build clamscan without building clamd,
clamdscan, clamav-milter, etc. We don't live in that world anymore and
certainly don't test that configuration.
To get the NotifyClamd freshclam feature to work with CMake builds we
either have to define BUILD_CLAMD from CMake as well, or remove the
feature entirely. I chose to remove it, because we don't test builds
without BUILD_CLAMD enabled and it adds needless complexity to our build
system and source.
If libcheck is missing, the unit tests are skipped. Header include path
changes mean clamav.h is not available when the unit tests are skipped,
and thus the check_clamav_skip.c files won't compile. This patch
removes the clamav.h dependency so `make check` will "work".
Default --wait timeout adjusted from 29 to 30 seconds.
--ping and --wait should exit with CL_ETIMEOUT (21) on timeout.
--ping should only return exit code 0 if clamd responds.
Silenced a couple switch fall-through warnings.
Added proc_fd_fname stack buffer to use with readlink, because the
pointers are restricted and using the same buffer with readlink could
result in undefined behavior.
Relocated clamonacc log verbosity initialization so early verbose log
messages will be printed.
Added a new status code for clamonacc startup checks so the --ping
feature can exit the process early with exit code 0.
Update the NEWS to add and correct content prior to the release
candidate.
Changed the version string to have the -rc suffix.
Also fixed a couple of --help and manpage issues.
Ping interval was not validated properly, causing a crash when the colon
separator was not present between attempts and interval. This was present
in clamonacc, as well as clamdscan.
This patch adds experimental-quality CMake build tooling.
The libmspack build required a modification to use "" instead of <> for
header #includes. This will hopefully be included in the libmspack
upstream project when adding CMake build tooling to libmspack.
Removed use of libltdl when using CMake.
Flex & Bison are now required to build.
If -DMAINTAINER_MODE, then GPERF is also required, though it currently
doesn't actually do anything. TODO!
I found that the autotools build system was generating the lexer output
but not actually compiling it, instead using previously generated (and
manually renamed) lexer c source. As a consequence, changes to the .l
and .y files weren't making it into the build. To resolve this, I
removed generated flex/bison files and fixed the tooling to use the
freshly generated files. Flex and bison are now required build tools.
On Windows, this adds a dependency on the winflexbison package,
which can be obtained using Chocolatey or may be manually installed.
CMake tooling only has partial support for building with external LLVM
library, and no support for the internal LLVM (to be removed in the
future). I.e. The CMake build currently only supports the bytecode
interpreter.
Many files used include paths relative to the top source directory or
relative to the current project, rather than relative to each build
target. Modern CMake support requires including internal dependency
headers the same way you would external dependency headers (albeit
with "" instead of <>). This meant correcting all header includes to
be relative to the build targets and not relative to the workspace.
For example, ...
```c
include "../libclamav/clamav.h"
include "clamd/clamd_others.h"
```
... becomes:
```c
// libclamav
include "clamav.h"
// clamd
include "clamd_others.h"
```
Fixes header name conflicts by renaming a few of the files.
Converted the "shared" code into a static library, which depends on
libclamav. The ironically named "shared" static library provides
features common to the ClamAV apps which are not required in
libclamav itself and are not intended for use by downstream projects.
This change was required for correct modern CMake practices but was
also required to use the automake "subdir-objects" option.
This eliminates warnings when running autoreconf which, in the next
version of autoconf & automake are likely to break the build.
libclamav used to build in multiple stages where an earlier stage is
a static library containing utils required by the "shared" code.
Linking clamdscan and clamdtop with this libclamav utils static lib
allowed these two apps to function without libclamav. While this is
nice in theory, the practical gains are minimal and it complicates
the build system. As such, the autotools and CMake tooling was
simplified for improved maintainability and this feature was thrown
out. clamdtop and clamdscan now require libclamav to function.
Removed the nopthreads version of the autotools
libclamav_internal_utils static library and added pthread linking to
a couple apps that may have issues building on some platforms without
it, with the intention of removing needless complexity from the
source. Kept the regular version of libclamav_internal_utils.la
though it is no longer used anywhere but in libclamav.
Added an experimental doxygen build option which attempts to build
clamav.h and libfreshclam doxygen html docs.
The CMake build tooling also may build the example program(s), which
isn't a feature in the Autotools build system.
Changed C standard to C90+ due to inline linking issues with socket.h
when linking libfreshclam.so on Linux.
Generate common.rc for win32.
Fix tabs/spaces in shared Makefile.am, and remove vestigial ifndef
from misc.c.
Add CMake files to the automake dist, so users can try the new
CMake tooling w/out having to build from a git clone.
clamonacc changes:
- Renamed FANOTIFY macro to HAVE_SYS_FANOTIFY_H to better match other
similar macros.
- Added a new clamav-clamonacc.service systemd unit file, based on
the work of ChadDevOps & Aaron Brighton.
- Added missing clamonacc man page.
Updates to clamdscan man page, add missing options.
Remove vestigial CL_NOLIBCLAMAV definitions (all apps now use
libclamav).
Rename Windows mspack.dll to libmspack.dll so all ClamAV-built
libraries have the lib-prefix with Visual Studio as with CMake.
Real-path checks are still needed in clamdscan when doing fd-passing and
streaming. This commit remedies that and improves some of the error
handling.
In addition, some cleanup to eliminate warnings on Windows added to the
shared code.
This patch relocates the real-path check from clamdscan and clamonacc
to clamd. While clamonacc is unlikely to send directories or symlinks
to be scanned, clamdscan may send directories. Real-path checks have
to be performed on the files, not the directories -- both because the
directories may contain symlinks and because the cli_realpath()
function wasn't written to support directories on Windows.
Using file type recognition scan mode for disk images and other raw
archive formats is problematic. One simple reason is that the contained
files will be detected and parsed and scanned twice, first when deteced
by the type recog scan, and later when the archive is extracted and the
files are properly scanned. Another reason is an increased likelihood
for incorrect type recognition, as seen with supposed MHTML files (they
weren't) found in GPT disk images.
Though a previous patch disabled embedded type recognition for GPT
files, this one extens this to the following:
- CL_TYPE_CPIO_OLD
- CL_TYPE_ZIP
- CL_TYPE_OLD_TAR
- CL_TYPE_POSIX_TAR
ZIP is included because file entries in a ZIP are incorrectly detected
as ZIPSFX's and though we also ensure not to scan ZIPSFX's found in
ZIP's, it's more efficient not to do the type recognition in the first
place and it prevents us from adding those bogus ZIPSFX entries into the
scan properties JSON.
This patch also fixes what appears to be a copy-paste typo, where
CL_TYPE_ISHIELD_MSI types were accidentally having their container value
set to CL_TYPE_AUTOIT.
Exit early from VBA scanning loop if virus found.
Add VBA/XLM suffix to ContainsMacros heuristics.
Fix setting status code for error and virus conditions.
Increment/decrement recursion counter when scanning vba dir.
Notably the commit adds a heuristic alert when VBA is extracted using
the new VBA extraction code and similarly adds "HasMacros":true to the
JSON scan properties.
In addition, a change was added to the cli_sanitize_filepath() function
so it converts posix pathseps to Windows pathseps on Windows and also
outputs a sanitized basename pointer (optional) which is used when
generating a temporary filename so that using a prefix with pathseps in
it won't cause file creation failures (observed with --leave-temps where
original filenames are incorporated into temporarily filenames).
Included soem error handling improvements for cli_vba_scandir() to
better track alert and macro detections.
Downgraded utf8 conversion error messages to debug messages because they
are too verbose in files with invalid filenames (observed in some
malware).
Changed the xlm macro and vba project temp filenames to include
"xlm_macros" and "vba_project" prefix, to make it easier to find them.
Relocated XLM and VBA temp files from the top-level tmp directory to the
current sub_tmpdir, so tempfiles for a given scan are more organized.
Fix an infinite loop in the new XLM macro parser.
Fix error handling, resource cleanup in OLE2 parser.
Fix issues tracking detected "viruses" in VBA & OLE2 parsers affecting
non-allmatch (regular) scan mode, wherein multiple viruses may be found
but each record lost and the overall detection comes up clean.
Also silence switch() fall-through warning for WORD/PPT/XL/HWP (OOXML)
file type fall-throughs to the ZIP parser (because they are zips).
Also silence switch() fall-through warning when handling the limits-
exceeded error types, checking for the limits-exceeded heuristic, and
continuing on to bail out with a clean verdict.
Changes cli_checkfp_virus to a recursive function which checks all
parent fmaps in the context for false positives
Simplifies params needed for cli_checkfp_virus to use the current digest
and fmap length that resides within the fmap struct itself
Add missing ping_clamd() declaration in client.h
Fix check for ping option to first check if ping option is NULL before
strdup'ing and checking if the alloc failed.
Fix format string for uint64_t print.
Correctly assign name pointer to stack buffer in cpio parser.
Remove vestigial variables from insert_list() function matcher-ac.c,
left over from before the load-time optimizations completely
restructured everything.
Silence warnings about unused parameters in progress bar callback
function.
Valgrind reports uninitialized `tag` stack buffer being used. While this
appears to be a false positive, it can't hurt to initialize this and
similar buffers in this function.
Fixes bound checks in recently rewritten VBA parser code (i.e. issue
does not affect prior versions).
Also improves VBA terminator header parsing to better match the spec,
per recommendation by Jonas Zaddach.
The clamonacc command doesn't present a `--debug` flag, but according
to your blog https://blog.clamav.net/2019/09/understanding-and-transitioning-to.html
the correct flag should be `--verbose`:
"[...]This is akin to clamd’s or clamscan’s --debug option, but isn’t
quite so noisy as either of those. By default, clamonacc does not
print any output after daemonizing, so you will have to pair this
option with --log or --foreground to use it.[...]"
- Updated `SUMHEAD` string macro with corrected alignment of `DBVER`
and `DBTIME` labels to accommodate current (wider) ClamAV version
numbers.
- Tweaked printing of primary thread counts and header values so
they fit in 80-column windows (as best as possible; the `h` suffix
of the `DBTIME` hour value doesn't fit, but at least the value
does). Previously, an 83-column window was required for the
information to fit.
- Make another line truncate at screen edge instead of 2 cols short.
- Make print_con_info truncate at screen edge and properly handle
messages with a newline.
- Use strrchr in print_con_info; handle wrapped lines in task list.
- Improve alignment of task list header labels.
- Fix null host/port bug; curses_inited should be initially 0.
- Fix compiler warning; fixes to line wrap patch; correct ENGINE
field width; a little tidy-up.
- Fix warnings; cleanup mem stats; version parsing fix; formatting tweaks
- Use newterm() and delscreen() for cleaner exit.
Also relocated codepage table from msdoc.h to entconv.h
Also adds new macros for codepages to reduce use of magic numbers when
referencing code pages elsewhere in libclamav.
294429: Negative check for fd_out occurs after a call to fdopen where
the value must not be negative. Coverity interprets this as a high
severity issue, even though it really isn't. Removing the needless check
should silence the false positive.
Somewhere between bison versions 3.5 and 3.6.4, having multiple
characters in a single-quoted string went from a warning to an error.
This commit corrects that by changing a literal to string,
Removed all autotools generates files. Autotools (autoconf, automake,
libtool, pkg-config, m4) will be required from now on for builds from
git clones.
Added autogen.sh to be run before ./configure.
Significant update to main .gitignore file.
Removed extraneous .gitignore files. A Git repository only needs one
.gitignore file.
Overflow check "(value >> 32) * 10 < INT32_MAX" may not work in
certain conditions, e.g. value is 0xcccccccdbcdc9cc
Note: This fixes oss-fuzz bug 16117.
Micah Snyder